ASF JIRA

ZooKeeper

ZOOKEEPER-728

make use of "protocolVersion" during session est

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

26/Mar/10 01:18

14/Dec/19 06:07

3.7.0

server

0

1

The current code does not make use of the "protocolVersion" field of the connect request.

We should increment this when making changes to the client protocol. This would allow the server to
accept/deny connections based on looking at the version supported by both the client and server. In some cases
multiple version combinations might be supported (version 1 or 2 or 3 from a client against a "version 3" server,
while v3 client against v2 server might fail). We should be sure to log this to the log4j log during connection (debug).
This might get complex, but if we don't support some combination(s) we should deny the connection and log it
rather than failing later in some unexpected way.

70737

No Perforce job exists for this issue.

0

42160

10 years, 6 days ago

0|i07kxz:

ZooKeeper

ZOOKEEPER-727

roadmap needs to be updated re our backward compatibility guarantees

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

25/Mar/10 18:54

14/Dec/19 06:06

3.7.0

documentation

0

1

The roadmap page is out of date wrt our current b/w compatibility rules. We need to update this.
http://wiki.apache.org/hadoop/ZooKeeper/Roadmap

additionally we need to include some of this detail in the release notes.

70782

No Perforce job exists for this issue.

0

32902

10 years, 1 week ago

0|i05ztj:

ZooKeeper

ZOOKEEPER-726

ZOOKEEPER-725 Performance and stress tests

Sub-task

Open

Major

Unresolved

Unassigned

Henry Robinson

25/Mar/10 02:05

0

2

We'd greatly benefit from long-running performance and stress tests that ran, say, nightly.

One thing I've done in the past is automatically run a set of classes with profiling enabled and periodically captured telemetry. Would be great to run something like this regularly as well, if the Hudson infrastructure were capable of doing so.

testing

214160

No Perforce job exists for this issue.

0

42161

10 years, 1 week ago

0|i07ky7:

ZooKeeper

ZOOKEEPER-725

Test organization, methodology, and infrastructure improvements

Test

Open

Major

Unresolved

Unassigned

Konstantin I Boudnik

23/Mar/10 22:08

31/Mar/10 01:16

build, tests

0

2

ZOOKEEPER-726

ZOOKEEPER-724

This is an umbrella feature for a number of improvements needed to happen in ZK test realm

214159

No Perforce job exists for this issue.

0

42162

24/Mar/10 2:15 AM

10 years, 1 day ago

0|i07kyf:

ZooKeeper

ZOOKEEPER-724

Improve junit test integration - log harness information

Improvement

Closed

Major

Fixed

Patrick D. Hunt

23/Mar/10 17:54

23/Nov/11 14:22

31/Mar/10 00:26

3.4.0

tests

0

HADOOP-6666, ZOOKEEPER-725

Currently we have ad-hoc logging (log4j) of setup/teardown and test start/end. Without this information it's often difficult to determine
what's going on (major milestones) in the logs relative to the tests performed.

In junit 4.x "@Rule" annotations have been added that allow this to be done much more easily. This patch will add this.

In particular we can access the test name easily during setup/teardown and the test itself. Additionally we can log more information on
success/failure of each test.

47633

No Perforce job exists for this issue.

1

33403

24/Mar/10 2:18 AM

9 years, 51 weeks, 6 days ago

Reviewed

0|i062wv:

ZooKeeper

ZOOKEEPER-723

ephemeral parent znodes

New Feature

Resolved

Major

Duplicate

Daniel Gómez Ferro

Benjamin Reed

23/Mar/10 13:32

19/Mar/19 09:14

server

7

15

ZOOKEEPER-834, ZOOKEEPER-2163

ephemeral znodes have the nice property of automatically cleaning up after themselves when the creator goes away, but since they can't have children it is hard to build subtrees that will cleanup after the clients that are using them are gone.

rather than changing the semantics of ephemeral nodes, i propose ephemeral parents: znodes that disappear when they have no more children. this cleanup would happen automatically when the last child is removed. an ephemeral parent is not tied to any particular session, so even if the creator goes away, the ephemeral parent will remain as long as there are children.

the when an ephemeral parent is created it will have an initial child, so that it doesn't get immediately removed. i think this child should be an ephemeral znode with a predefined name, "firstChild".

container_znode_type

71038

No Perforce job exists for this issue.

2

42163

06/Aug/10 3:13 PM

1 year, 2 days ago

0|i07kyn:

ZooKeeper

ZOOKEEPER-722

zkServer.sh uses sh's builtin echo on BSD, behaves incorrectly.

Bug

Closed

Minor

Fixed

Ivan Kelly

22/Mar/10 10:49

23/Nov/11 14:22

12/Apr/10 02:33

3.3.0

3.3.1, 3.4.0

scripts

0

1

Mac OS X

zkServer.sh output the PID of the zookeeper process with:
echo -n $! > "$ZOOPIDFILE"

This uses -n which sh's builtin echo does not support. From echo's manpage.
<snip>
Some shells may provide a builtin echo command which is similar or identical to this utility. Most notably, the builtin echo in sh(1) does not accept
the -n option. Consult the builtin(1) manual page.
</snip>

This means that echo -n PID > ZOOPIDFILE will mean the contents of ZOOPIDFILE will be "-n PID". This stops zkServer.sh stop from working correctly.

47634

No Perforce job exists for this issue.

1

32903

22/Mar/10 11:13 AM

9 years, 50 weeks, 3 days ago

Fix zkServer.sh script for BSD systems.

Reviewed

0|i05ztr:

ZooKeeper

ZOOKEEPER-721

Minor cleanup related to the log4j version change from 1.2.15 -> 1.2.16

Improvement

Resolved

Minor

Fixed

Sean Busbey

Paolo Castagna

21/Mar/10 04:25

12/Dec/12 06:05

12/Dec/12 02:32

3.3.0

3.5.0

build

0

2

The generated Maven pom.xml file contains:

{code}
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
<scope>compile</scope>
</dependency>
{code}

It would be better to have:

{code}
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
</exclusion>
<exclusion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
</exclusion>
<exclusion>
<groupId>javax.mail</groupId>
<artifactId>mail</artifactId>
</exclusion>
</exclusions>
</dependency>
{code}

This would avoid to people depending on ZooKeeper artifacts the pain to exclude such "broken" dependencies by themselves.

A similar issue is discussed in HADOOP-6629, although I am not clear on how to solve fix it using Ivy.

ivy, maven

70727

No Perforce job exists for this issue.

1

2354

22/Apr/10 3:17 AM

7 years, 15 weeks, 1 day ago

Reviewed

0|i00ran:

ZooKeeper

ZOOKEEPER-720

Use zookeeper-{version}-sources.jar instead of zookeeper-{version}-src.jar to publish sources in the Maven repository

Bug

Closed

Trivial

Fixed

Paolo Castagna

21/Mar/10 04:04

23/Nov/11 14:22

12/Apr/10 02:19

3.3.0

3.3.1, 3.4.0

build

0

1

The artifact with the sources to be published in the Maven repository should be named ${artifactId}-${version}-sources.jar not ${artifactId}-${version}-src.jar.

See also: http://maven.apache.org/guides/mini/guide-central-repository-upload.html and ZOOKEEPER-224

maven

47635

No Perforce job exists for this issue.

1

32904

21/Mar/10 4:39 AM

9 years, 50 weeks, 3 days ago

Reviewed

0|i05ztz:

ZooKeeper

ZOOKEEPER-719

Add throttling to BookKeeper client

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

19/Mar/10 13:35

23/Nov/11 14:22

09/Jul/10 16:54

3.3.0

3.4.0

contrib-bookkeeper

0

1

Add throttling to client to control the rate of operations to bookies.

47636

No Perforce job exists for this issue.

4

32905

22/Mar/10 6:55 PM

9 years, 36 weeks, 1 day ago

0|i05zu7:

ZooKeeper

ZOOKEEPER-718

the fatjar is missing libraries

Bug

Closed

Major

Fixed

Benjamin Reed

19/Mar/10 12:09

26/Mar/10 13:25

19/Mar/10 13:28

3.3.0

0

2

when we moved to ivy, we didn't update the fatjar build.xml to grab libraries out of the new location that ivy uses for downloaded libraries.

47637

No Perforce job exists for this issue.

1

32906

19/Mar/10 1:28 PM

10 years, 1 week, 5 days ago

Reviewed

0|i05zuf:

ZooKeeper

ZOOKEEPER-717

add a preferred list to the instancemanager

Improvement

Closed

Major

Fixed

Benjamin Reed

19/Mar/10 12:04

26/Mar/10 13:25

19/Mar/10 13:30

3.3.0

tests

0

1

it would be nice to be able to list the preferred containers to assign tasks to when using instance containers. right now assignments are made in hashmap order which is more or less psuedo-random.

47638

No Perforce job exists for this issue.

1

33404

19/Mar/10 12:36 PM

10 years, 1 week, 5 days ago

Reviewed

0|i062x3:

ZooKeeper

ZOOKEEPER-716

dump server memory detail to the log during startup

Improvement

Resolved

Minor

Fixed

Michi Mutsuzaki

Patrick D. Hunt

18/Mar/10 14:02

21/May/14 18:54

20/May/14 11:12

3.5.0

server

0

3

dump any/all memory information that's available from system/runtime during the server startup (same code where we output the env information)

helps with debugging

70754

No Perforce job exists for this issue.

1

42164

15/May/14 6:59 PM

5 years, 44 weeks, 1 day ago

0|i07kyv:

ZooKeeper

ZOOKEEPER-715

add better reporting for initLimit being reached

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

18/Mar/10 13:43

05/Feb/20 07:16

3.2.2

3.7.0, 3.5.8

server

0

2

ZOOKEEPER-713

when the initLimit is reached (sending zk database to follower) we don't print out very useful log information.

we need to add details on when the snap is read, when the new snap is written, and the start/end of the transfer to follower. start/end should include useful logging in the case that initLimit is reached and the operation fails.

70729

No Perforce job exists for this issue.

0

42165

04/May/10 5:56 PM

43 weeks, 1 day ago

0|i07kz3:

ZooKeeper

ZOOKEEPER-714

snapshotting doesn't handle runtime exceptions (like out of memory) well

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

18/Mar/10 13:38

05/Feb/20 07:16

3.2.2

3.7.0, 3.5.8

server

0

2

ZOOKEEPER-713

If an out of memory occurs during snapshotting we recover from the error, however we should be more explict about the problem.

Currently we log a warning, really we should log a FATAL and exit. This would force the server to restart (supervisor) and make it very clear to the user there is a problem. In some cases users don't notice the OOM exception (we have trouble noticing it sometimes ourselves) as it doesn't stand out on the logs. Exiting would be more explicit. However since we'd like to remove sysexits due to container issues it's not clear what we should do here in addition to making this FATAL.

70723

No Perforce job exists for this issue.

0

32907

10 years, 2 weeks ago

0|i05zun:

ZooKeeper

ZOOKEEPER-713

zookeeper fails to start - broken snapshot?

Bug

Closed

Major

Invalid

Unassigned

Lukasz Osipiuk

18/Mar/10 11:53

26/Mar/10 13:20

18/Mar/10 13:41

3.2.2

0

ZOOKEEPER-714, ZOOKEEPER-715

debian lenny; ia64; xen virtualization

Hi guys,

The following is not a bug report but rather a question - but as I am attaching large files I am posting it here rather than on mailinglist.

Today we had major failure in our production environment. Machines in zookeeper cluster gone wild and all clients got disconnected.
We tried to restart whole zookeeper cluster but cluster got stuck in leader election phase.

Calling stat command on any machine in the cluster resulted in 'ZooKeeperServer not running' message
In one of logs I noticed 'Invalid snapshot' message which disturbed me a bit.

We did not manage to make cluster work again with data. We deleted all version-2 directories on all nodes and then cluster started up without problems.
Is it possible that snapshot/log data got corrupted in a way which made cluster unable to start?
Fortunately we could rebuild data we store in zookeeper as we use it only for locks and most of nodes is ephemeral.

I am attaching contents of version-2 directory from all nodes and server logs.
Source problem occurred some time before 15. First cluster restart happened at 15:03.
At some point later we experimented with deleting version-2 directory so I would not look at following restart because they can be misleading due to our actions.

I am also attaching zoo.cfg. Maybe something is wrong at this place.
As I know look into logs i see read timeout during initialization phase after 20secs (initLimit=10, tickTime=2000).
Maybe all I have to do is increase one or other. which one? Are there any downsides of increasing tickTime.

Best regards, Łukasz Osipiuk

PS. due to attachment size limit I used split. to untar use
cat nodeX-version-2.tgz-* |tar -xz

214158

No Perforce job exists for this issue.

12

32908

18/Mar/10 12:23 PM

10 years, 1 week, 6 days ago

0|i05zuv:

ZooKeeper

ZOOKEEPER-712

Bookie recovery

New Feature

Closed

Major

Fixed

Erwin Tam

Flavio Paiva Junqueira

18/Mar/10 11:39

23/Nov/11 14:22

18/Feb/11 23:01

3.3.3, 3.4.0

contrib-bookkeeper

0

BOOKKEEPER-2

Recover the ledger fragments of a bookie once it crashes.

47639

No Perforce job exists for this issue.

2

33405

16/Jun/10 6:28 PM

9 years, 6 weeks ago

Reviewed

0|i062xb:

ZooKeeper

ZOOKEEPER-710

permanent ZSESSIONMOVED error after client app reconnects to zookeeper cluster

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

Lukasz Osipiuk

18/Mar/10 07:39

26/Mar/10 13:25

19/Mar/10 13:19

3.2.2

3.2.3, 3.3.0

server

0

3

debian lenny; ia64; xen virtualization

Originally problem was described on Users mailing list starting with this [post|http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/201003.mbox/<3b910d891003160743k38e2e7c9y830b182d88396d55@mail.gmail.com>].
Below I restate it in more organized form.

We occasionally (few times a day) observe that our client application disconnects from Zookeeper cluster.
Application is written in C++ and we are using libzookeeper_mt library. In version 3.2.2.

The disconnects we are observing are probably related to some problems with our network infrastructure - we are observing periods with great packet loss between machines in our DC.

Sometimes after client application (i.e. zookeeper library) reconnects to zookeeper cluster we are observing that all subsequent requests return ZSESSIONMOVED error. Restarting client app helps - we always pass 0 as clientid to zookeeper_init function so old session is not reused.

On 16-03-2010 we observed few occurences of problem. Example ones:
- 22:08; client IP 10.1.112.60 (app1); sessionID 0x22767e1c9630000
- 14:21; client IP 10.1.112.61 (app2); sessionID 0x324dcc1ba580085

I attach logs of cluster and application nodes (only stuff concerining zookeeper):
- [^zookeeper-node1.log.2010-03-16.gz] - logs of zookeepr cluster node 1 10.1.112.62
- [^zookeeper-node2.log.2010-03-16.gz] - logs of zookeepr cluster node 2 10.1.112.63
- [^zookeeper-node3.log.2010-03-16.gz] - logs of zookeepr cluster node 3 10.1.112.64
- [^app1.log.2010-03-16.gz] - application logs of app1 10.1.112.60
- [^app2.log.2010-03-16.gz] - application logs of app2 10.1.112.61

I also made some analysis of case at 22:08:
- Network glitch which resulted in problem occurred at about 22:08.
- From what I see since 17:48 node2 was the leader and it did not
change later yesterday.
- Client was connected to node2 since 17:50
- At around 22:09 client tried to connect to every node (1,2,3).
Connections to node1 and node3 were closed
with exception "Exception causing close of session 0x22767e1c9630000
due to java.io.IOException: Read error".
Connection to node2 stood alive.
- All subsequent operations were refused with ZSESSIONMOVED error.
Error visible both on client and on server side.

47640

No Perforce job exists for this issue.

7

32909

18/Mar/10 1:45 PM

10 years, 1 week, 5 days ago

Reviewed

0|i05zv3:

ZooKeeper

ZOOKEEPER-709

bookkeeper build failing with missing factory

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

18/Mar/10 01:57

26/Mar/10 13:25

18/Mar/10 02:23

3.3.0

contrib-bookkeeper

0

2

ant test in bookkeeper results in

compile-test:
[javac] Compiling 10 source files to /home/phunt/dev/workspace/gitzk/build/contrib/bookkeeper/test
[javac] /home/phunt/dev/workspace/gitzk/src/contrib/bookkeeper/test/org/apache/bookkeeper/test/BaseTestCase.java:91: cannot find symbol
[javac] symbol : constructor Factory(java.lang.Integer)
[javac] location: class org.apache.zookeeper.server.NIOServerCnxn.Factory
[javac] serverFactory = new NIOServerCnxn.Factory(ZooKeeperDefaultPort);
[javac] ^
[javac] 1 error

Flavio can you take a look at this one? (patch)

47641

No Perforce job exists for this issue.

1

32910

18/Mar/10 2:12 AM

10 years, 2 weeks ago

Reviewed

0|i05zvb:

ZooKeeper

ZOOKEEPER-708

zkpython failing due to undefined symbol deallocate_String_vector

Bug

Closed

Blocker

Fixed

Mahadev Konar

Patrick D. Hunt

18/Mar/10 01:34

26/Mar/10 13:25

18/Mar/10 02:48

3.3.0

c client, contrib-bindings

0

2

ant test in zkpython is failing. I think this is due to mahadev's changes to remove unnecessary exports from the client lib.

[exec] ImportError: /home/phunt/dev/workspace/gitzk/build/contrib/zkpython/lib.linux-x86_64-2.6/zookeeper.so: undefined symbol: deallocate_String_vector

Mahadev can you take a look?

47642

No Perforce job exists for this issue.

3

32911

18/Mar/10 1:49 AM

10 years, 2 weeks ago

Reviewed

0|i05zvj:

ZooKeeper

ZOOKEEPER-707

c client close can crash with cptr null

Bug

Open

Critical

Unresolved

Mahadev Konar

Patrick D. Hunt

17/Mar/10 20:21

14/Dec/19 06:07

3.3.0

3.7.0

c client

0

ZOOKEEPER-768, ZOOKEEPER-804

saw this in the zktest_mt at the end of 3.3.0, seems unlikely to happen though as it only failed after running the test 10-15 times.

Zookeeper_simpleSystem::testAuth ZooKeeper server started : elapsed 26011 : OK
Zookeeper_simpleSystem::testHangingClientzktest-mt: src/zookeeper.c:1950: zookeeper_process: Assertion `cptr' failed.
Aborted

71230

No Perforce job exists for this issue.

0

32912

04/May/10 5:53 PM

8 years, 41 weeks, 1 day ago

0|i05zvr:

ZooKeeper

ZOOKEEPER-706

large numbers of watches can cause session re-establishment to fail

Bug

Closed

Critical

Fixed

Chris Thunes

Patrick D. Hunt

17/Mar/10 14:08

01/Nov/18 03:01

12/Jun/15 20:50

3.1.2, 3.2.2, 3.3.0

3.4.7, 3.5.2, 3.6.0

c client, java client

7

30

0

3000

ZOOKEEPER-1162, ZOOKEEPER-1644

If a client sets a large number of watches the "set watches" operation during session re-establishment can fail.

for example:
WARN [NIOServerCxn.Factory:22801:NIOServerCnxn@417] - Exception causing close of session 0xe727001201a4ee7c due to java.io.IOException: Len error 4348380

in this case the client was a web monitoring app and had set both data and child watches on > 32k znodes.

there are two issues I see here we need to fix:

1) handle this case properly (split up the set watches into multiple calls I guess...)
2) the session should have expired after the "timeout". however we seem to consider any message from the client as re-setting the expiration on the server side. Probably we should only consider messages from the client that are sent during an established session, otherwise we can see this situation where the session is not established however the session is not expired either. Perhaps we should create another JIRA for this particular issue.

100%

3000

0

pull-request-available

62980

No Perforce job exists for this issue.

5

32913

15/Jun/11 2:09 AM

1 year, 29 weeks, 2 days ago

0|i05zvz:

ZooKeeper

ZOOKEEPER-705

Fails to Build due to unknown opcode 'lock' in mt_adaptor.c

Bug

Resolved

Minor

Duplicate

Thomas Koch

14/Mar/10 16:44

05/Jun/18 16:14

10/Dec/13 16:25

3.2.2

3.5.0

c client

0

3

The Debian package of Zookeeper[1] fails to build on some architectures, see below. The issue is an unknown assembler opcode. The Bugs is reported in the debian bugtracker as #568618[2]. A patch is available. I've uploaded it here too for your convenience.

[1] http://packages.qa.debian.org/z/zookeeper.html
[2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=568618

Buildd status overview:
https://buildd.debian.org/status/package.php?p=zookeeper

Some Failed Buildd Logs:
Sparc: https://buildd.debian.org/fetch.cgi?pkg=zookeeper;ver=3.2.2%2Bdfsg3-2;arch=sparc;stamp=1265466795
S390: https://buildd.debian.org/fetch.cgi?pkg=zookeeper;ver=3.2.2%2Bdfsg3-2;arch=s390;stamp=1265415637
powerpc: https://buildd.debian.org/fetch.cgi?pkg=zookeeper;ver=3.2.2%2Bdfsg3-2;arch=powerpc;stamp=1266677031
mipsel: https://buildd.debian.org/fetch.cgi?pkg=zookeeper&arch=mipsel&ver=3.2.2%2Bdfsg3-2&stamp=1268124320&file=log&as=raw

67867

No Perforce job exists for this issue.

1

32914

14/Mar/10 11:50 PM

1 year, 41 weeks, 2 days ago

0|i05zw7:

ZooKeeper

ZOOKEEPER-704

GSoC 2010: Read-Only Mode

Wish

Open

Major

Unresolved

Sergey Doroshenko

Henry Robinson

12/Mar/10 19:59

27/Jan/11 12:45

0

6

ZOOKEEPER-784, ZOOKEEPER-827, ZOOKEEPER-830

ZOOKEEPER-769

Read-only mode
Possible Mentor
Henry Robinson (henry at apache dot org)

Requirements
Java and TCP/IP networking

Description
When a ZooKeeper server loses contact with over half of the other servers in an ensemble ('loses a quorum'), it stops responding to client requests because it cannot guarantee that writes will get processed correctly. For some applications, it would be beneficial if a server still responded to read requests when the quorum is lost, but caused an error condition when a write request was attempted.

This project would implement a 'read-only' mode for ZooKeeper servers (maybe only for Observers) that allowed read requests to be served as long as the client can contact a server.

This is a great project for getting really hands-on with the internals of ZooKeeper - you must be comfortable with Java and networking otherwise you'll have a hard time coming up to speed.

gsoc, mentor

214157

No Perforce job exists for this issue.

0

42166

08/May/10 9:26 AM

9 years, 9 weeks ago

0|i07kzb:

ZooKeeper

ZOOKEEPER-703

GSoC 2010: ZooKeeper DNS Server

Wish

Open

Major

Unresolved

Unassigned

Henry Robinson

12/Mar/10 19:56

13/Mar/16 18:30

0

8

ZooKeeper DNS Server
Possible Mentor
Henry Robinson (henry at apache dot org)

Requirements
Java or Python or C

Description
Although ZooKeeper is primarily used for co-ordination of distributed processes, its consistency semantics means that it's a good candidate for serving small (key,value) records as well. The Domain Name Service has similar requirements, raising the interesting question of whether ZooKeeper would be a capable DNS server for your local network. One intriguing possibility is having versioned DNS records, such that known-good configurations can be stored and rolled back to in the case of an issue. If this versioning primitive proves to be useful, it's easy to imagine other types of configuration that could be stored.

This project would involve designing and building an RFC-1035 compliant DNS server and performing a detailed performance study against an already existant simple DNS server like tinydns.

gsoc, mentor

214156

No Perforce job exists for this issue.

0

42167

22/Mar/10 4:22 PM

4 years, 1 week, 4 days ago

0|i07kzj:

ZooKeeper

ZOOKEEPER-702

GSoC 2010: Failure Detector Model

Wish

Open

Major

Unresolved

Abmar Barros

Henry Robinson

12/Mar/10 19:44

14/Dec/19 06:09

3.7.0

0

7

ZOOKEEPER-810, ZOOKEEPER-811, ZOOKEEPER-812

HDFS-779, ZOOKEEPER-823, HBASE-5843

Failure Detector Module
Possible Mentor
Henry Robinson (henry at apache dot org)

Requirements
Java, some distributed systems knowledge, comfort implementing distributed systems protocols

Description
ZooKeeper servers detects the failure of other servers and clients by counting the number of 'ticks' for which it doesn't get a heartbeat from other machines. This is the 'timeout' method of failure detection and works very well; however it is possible that it is too aggressive and not easily tuned for some more unusual ZooKeeper installations (such as in a wide-area network, or even in a mobile ad-hoc network).

This project would abstract the notion of failure detection to a dedicated Java module, and implement several failure detectors to compare and contrast their appropriateness for ZooKeeper. For example, Apache Cassandra uses a phi-accrual failure detector (http://ddsg.jaist.ac.jp/pub/HDY+04.pdf) which is much more tunable and has some very interesting properties. This is a great project if you are interested in distributed algorithms, or want to help re-factor some of ZooKeeper's internal code.

gsoc, mentor

43

No Perforce job exists for this issue.

35

42168

03/Apr/10 4:19 AM

6 years, 24 weeks, 3 days ago

0|i07kzr:

ZooKeeper

ZOOKEEPER-701

GSoC 2010: Monitoring Recipes and Web-based Administrative Interface

Wish

Resolved

Major

Fixed

Andrei Savu

Henry Robinson

12/Mar/10 19:42

18/Aug/10 02:59

16/Aug/10

0

4

ZOOKEEPER-744, ZOOKEEPER-799, ZOOKEEPER-808, ZOOKEEPER-809

Monitoring Recipes And Web-based Administrative Interface
Mentor: Patrick Hunt (phunt@apache.org)

Requirements:
Modern web platform - e.g. Django. Some design or UI skills would help. Java for adding methods to ZooKeeper.

Description:
ZooKeeper is a complex distributed system. Understanding how well it is running is tremendously important. Patrick Hunt has created a Django-based dashboard (see http://github.com/phunt/zookeeper_dashboard) that allows some insight into how ZooKeeper is running. This is a great foundation on which to build; however there are improvements that could be made! This project would capture much more information from ZooKeeper, adding hooks to retrieve it where necessary and visualise it in a appealing and useful way. Integration with Ganglia would be a definite plus.

gsoc, mentor

47643

No Perforce job exists for this issue.

1

33406

29/Apr/10 6:38 PM

9 years, 32 weeks, 1 day ago

0|i062xj:

ZooKeeper

ZOOKEEPER-699

GSoC 2010: Optimizations for WAN Deployments

Wish

Open

Major

Unresolved

Unassigned

Henry Robinson

12/Mar/10 19:39

03/Apr/10 03:25

1

2

Optimizations for WAN Deployments

Possible Mentor
Henry Robinson (henry at apache dot org)

Requirements
Java, some networking familiarity

Description
ZK 3.3.0 added observers which are non-voting members of a ZK ensemble. One use case for observers is as a proxy to a remote voting ensemble, say in a different data center. Since observers do not need to vote, there are less strict latency requirements on the delivery of messages to them. WAN traffic is also expensive. This project would investigate and implement batching of messages to observers, and potential mechanisms for decreasing the number of messages that need to be sent. For example, a destructive update to a znode twice in a row does not theoretically need to be sent twice - although making this work correctly with ZAB will be a challenge.

gsoc, mentor

214154

No Perforce job exists for this issue.

0

42170

03/Apr/10 3:25 AM

10 years, 2 weeks, 5 days ago

0|i07l07:

ZooKeeper

ZOOKEEPER-698

intermittent JMX test failures due to not verifying QuorumPeer shutdown

Bug

Closed

Critical

Fixed

Patrick D. Hunt

12/Mar/10 17:22

26/Mar/10 13:25

12/Mar/10 19:07

3.3.0

tests

0

2

in some cases the tests are failing with JMX errors. From the logs I can see that QP was shutdown, however it did not exit it's thread until some time much later. This is causing interference with subsequent tests, causing the test to fail.

I have a patch that attempts to verify that the QP was shutdown (by joining the thread). It turns out that tests based on QuorumBase do this check (join) however some of the other tests do not. I believe this will address the issue.

47644

No Perforce job exists for this issue.

1

32915

12/Mar/10 5:59 PM

10 years, 2 weeks, 3 days ago

Reviewed

0|i05zwf:

ZooKeeper

ZOOKEEPER-697

TestQuotaQuorum is failing on Hudson

Bug

Resolved

Critical

Cannot Reproduce

Mahadev Konar

10/Mar/10 11:20

09/Oct/13 02:23

3.5.0

0

The hudson test build failed

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/729/testReport/junit/org.apache.zookeeper.test/QuorumQuotaTest/testQuotaWithQuorum/

71229

No Perforce job exists for this issue.

0

32916

11/Mar/10 4:00 PM

8 years, 41 weeks, 1 day ago

0|i05zwn:

ZooKeeper

ZOOKEEPER-696

NPE in the hudson logs, seems nioservercnxn closed twice

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

10/Mar/10 00:41

26/Mar/10 13:25

10/Mar/10 22:17

3.3.0

server

0

2

seeing the following on the console for http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/729/

looks like the cnxn is closed twice? (the second time 'sock' is null). perhaps it's due to client closing and sending session term, then closing socket, server sees the read return -1, so closes cnxn, then sees the session close request (which was queued)?

[junit] 2010-03-10 03:15:53,205 - INFO [main:NIOServerCnxn@1232] - Closed socket connection for client /127.0.0.1:41285 which had sessionid 0x127461233fc0000
[junit] 2010-03-10 03:15:53,206 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn$Factory@269] - Ignoring unexpected runtime exception
[junit] java.lang.NullPointerException
[junit] at org.apache.zookeeper.server.NIOServerCnxn.close(NIOServerCnxn.java:1232)
[junit] at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:594)
[junit] at org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:259)

47645

No Perforce job exists for this issue.

1

32917

10/Mar/10 6:11 PM

10 years, 3 weeks ago

Reviewed

0|i05zwv:

ZooKeeper

ZOOKEEPER-695

Need to remove the lookForLeader duplicated code in LENonTerminateTest

Bug

Open

Major

Unresolved

Unassigned

Benjamin Reed

09/Mar/10 20:25

10/Mar/10 13:55

0

ZOOKEEPER-684

To do the LENonTerminateTest we need to hook into the lookForLeader code. In the last release we did this by duplicating the code. This needs to be cleaned up.

214153

No Perforce job exists for this issue.

0

32918

10/Mar/10 2:56 AM

10 years, 3 weeks, 1 day ago

0|i05zx3:

ZooKeeper

ZOOKEEPER-694

Add unit tester for Zookeeper

Improvement

Open

Minor

Unresolved

Unassigned

David Rosenstrauch

09/Mar/10 17:58

14/Dec/19 06:08

3.7.0

c client, tests

0

It would be nice to have a mock/fake version of org.apache.zookeeper.Zookeeper that could be used for unit testing. i.e., a single instance Zookeeper that operates completely in memory, with no network or disk I/O.

This would make it possible to pass one of the memory-only fake Zookeeper's into unit tests, while using a real Zookeeper in production code.

i.e., maybe something like this:

public interface ZooKeeperService {
...
}

public class ZooKeeperTester implements ZooKeeperService {
...
(stand-alone, in-memory, test implementation)
...
}

public class ZooKeeper implements ZooKeeperService {
...
("real" implementation)
...
}

70796

No Perforce job exists for this issue.

0

42171

09/Mar/10 6:33 PM

10 years, 3 weeks, 2 days ago

0|i07l0f:

ZooKeeper

ZOOKEEPER-693

TestObserver stuck in tight notification loop in FLE

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

Henry Robinson

09/Mar/10 16:40

26/Mar/10 13:25

10/Mar/10 13:52

3.3.0

0

4

See http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/Zookeeper-Patch-h7.grid.sp2.yahoo.net/77/testReport/junit/org.apache.zookeeper.test/ObserverTest/testObserver/

[exec] [junit] 2010-03-04 00:23:37,803 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:11229:FastLeaderElection@683] - Notification time out: 3200
[exec] [junit] 2010-03-04 00:23:37,804 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:11229:FastLeaderElection@689] - Notification: 2, 0, 2, 3, LOOKING, LOOKING, 1

ad infinitum.

47646

No Perforce job exists for this issue.

2

32919

10/Mar/10 5:40 AM

10 years, 3 weeks ago

Reviewed

0|i05zxb:

ZooKeeper

ZOOKEEPER-692

upgrade junit to latest version (4.8.1)

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

09/Mar/10 16:11

26/Mar/10 13:25

10/Mar/10 13:15

3.3.0

build

0

1

upgrade junit to 4.8.1

47647

No Perforce job exists for this issue.

1

33407

09/Mar/10 4:35 PM

10 years, 3 weeks ago

Reviewed

0|i062xr:

ZooKeeper

ZOOKEEPER-691

Interface changed for NIOServer.Factory

Bug

Closed

Major

Fixed

Benjamin Reed

09/Mar/10 15:40

26/Mar/10 13:25

09/Mar/10 15:50

3.3.0

contrib-bookkeeper

0

2

BookKeeper starts a ZooKeeper server and needs to create an NIOServer.Factory, but the constructor changed.

47648

No Perforce job exists for this issue.

1

32920

09/Mar/10 3:46 PM

10 years, 3 weeks, 1 day ago

Reviewed

0|i05zxj:

ZooKeeper

ZOOKEEPER-690

AsyncTestHammer test fails on hudson.

Bug

Closed

Major

Cannot Reproduce

Henry Robinson

Mahadev Konar

08/Mar/10 20:42

23/Nov/11 14:22

16/Jul/11 13:49

3.4.0

0

1

ZOOKEEPER-764

the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.

67900

No Perforce job exists for this issue.

14

32921

17/Mar/10 1:44 AM

8 years, 36 weeks, 5 days ago

0|i05zxr:

ZooKeeper

ZOOKEEPER-689

release build broken - ivysettings.xml not copied during "package"

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

08/Mar/10 20:23

26/Mar/10 13:25

08/Mar/10 20:50

3.3.0

build

0

2

ivysettings.xml was added in 3.3.0 but it is not copied into the release artifact via ant "package" target of build.xml

47649

No Perforce job exists for this issue.

1

32922

08/Mar/10 8:26 PM

10 years, 3 weeks, 2 days ago

Reviewed

0|i05zxz:

ZooKeeper

ZOOKEEPER-688

explain session expiration better in the docs & faq

Bug

Closed

Critical

Fixed

Patrick D. Hunt

05/Mar/10 12:10

26/Mar/10 13:25

05/Mar/10 17:03

3.3.0

documentation

0

2

We are not clear enough (and the diagram we do have seems misleading) on _when_ session expirations are generated. In particular the fact that you only get expirations when the client is connected to the cluster, not when disconnected.

we need to detail:

1) when do you get expiration
2) what is the sequence of events that the watcher sees, from disco state, to getting the expiration (say the expiration happens when the client is disco, what do you see in the watcher while you are getting reconnected)
3) we need to give some examples of how to test this. We should be explicit that "pulling the network cable" on the client will not show expiration since the cliient will not be reconnected.

47650

No Perforce job exists for this issue.

2

32923

05/Mar/10 1:49 PM

10 years, 3 weeks, 5 days ago

Reviewed

0|i05zy7:

ZooKeeper

ZOOKEEPER-687

LENonterminatetest fails on some machines.

Bug

Closed

Major

Fixed

Mahadev Konar

04/Mar/10 17:12

26/Mar/10 13:25

04/Mar/10 17:56

3.3.0

0

2

LENonterminateTest fails with the following error:

{noformat}
2010-03-04 20:26:32,347 - INFO [Thread-0:LeaderElection@155] - Server address: 0.0.0.0/0.0.0.0:11223
2010-03-04 20:26:32,348 - WARN [Thread-0:LeaderElection@195] - Ignoring exception while looking for leader
java.io.IOException: Network is unreachable
at java.net.PlainDatagramSocketImpl.send(Native Method)
at java.net.DatagramSocket.send(DatagramSocket.java:612)
at org.apache.zookeeper.server.quorum.LeaderElection.lookForLeader(LeaderElection.java:169)
at org.apache.zookeeper.test.LENonTerminateTest$LEThread.run(LENonTerminateTest.java:83)
{noformat}

47651

No Perforce job exists for this issue.

2

32924

04/Mar/10 5:47 PM

10 years, 3 weeks, 6 days ago

Reviewed

0|i05zyf:

ZooKeeper

ZOOKEEPER-686

Add examples for atomic operations to recipes (docs & code)

Improvement

Open

Major

Unresolved

Unassigned

David Rosenstrauch

03/Mar/10 17:47

14/Dec/19 06:06

3.2.2

3.7.0

recipes

0

1

The zookeeper docs do a good job of explaining how to do low-level synchronization primitives with zookeeper. But it's not immediately obvious after reading the docs how to implement higher-level synchronization primitives - for example how to do zookeeper equivalents of the java Atomic concurrency primitives, like AtomicInteger.

So as per Patrick Hunt: "This is def. something we should add to the recipes (docs & code lib)"

70744

No Perforce job exists for this issue.

0

42172

10 years, 4 weeks, 1 day ago

zookeeper atomic

0|i07l0n:

ZooKeeper

ZOOKEEPER-685

make C system tests use multiple servers

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

03/Mar/10 13:57

14/Dec/19 06:09

3.7.0

c client

0

ZOOKEEPER-622

the C system tests run against a single server. we really need to use a multiple server configuration.

70762

No Perforce job exists for this issue.

1

42173

05/Mar/10 6:23 PM

10 years, 3 weeks, 6 days ago

0|i07l0v:

ZooKeeper

ZOOKEEPER-684

Race in LENonTerminateTest

Bug

Closed

Critical

Fixed

Henry Robinson

Flavio Paiva Junqueira

03/Mar/10 05:18

26/Mar/10 13:25

09/Mar/10 20:28

3.3.0

leaderElection, server

0

2

ZOOKEEPER-695

testNonTermination failed during a Hudson run for ZOOKEEPER-59. After inspecting the output, it looks like server is electing 2 as a leader and leaving. Given that 2 is just a mock server, server 0 remains alone in leader election.

47652

No Perforce job exists for this issue.

4

32925

05/Mar/10 5:50 PM

10 years, 3 weeks, 1 day ago

Reviewed

0|i05zyn:

ZooKeeper

ZOOKEEPER-683

LogFormatter fails to parse transactional log files

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

01/Mar/10 02:04

26/Mar/10 13:25

03/Mar/10 18:07

3.2.2

3.3.0

server

0

2

LogFormatter fails to parse txn log files - seems the tool was never updated to handle FileHeader.

It would be good to update the docs on txn log file to include detail on the file format.

47653

No Perforce job exists for this issue.

1

32926

01/Mar/10 9:09 PM

10 years, 4 weeks, 1 day ago

Reviewed

0|i05zyv:

ZooKeeper

ZOOKEEPER-682

Event is not processed when the watcher is set to watch "/" if chrooted

Bug

Closed

Blocker

Fixed

Scott Wang

25/Feb/10 07:01

26/Mar/10 13:25

03/Mar/10 20:29

3.2.2

3.3.0

java client

0

3

After the event notification response from server is received, the client will convert the server path to the client path if chrooted by:

event.setPath(serverPath.substring(chrootPath.length());

If chrootPath and serverPath are the same, then the event's path will be set to a null string.

But the key of the watcher's map is "/", not a null string, so the watcher will not get notified at all.

47654

No Perforce job exists for this issue.

2

32927

25/Feb/10 12:51 PM

10 years, 4 weeks ago

Reviewed

0|i05zz3:

ZooKeeper

ZOOKEEPER-681

Minor doc issue re unset maxClientCnxns

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

Vegard B. Havdal

25/Feb/10 06:25

26/Mar/10 13:25

04/Mar/10 21:27

3.2.2

3.3.0

documentation

0

2

Just a small issue, the doc says that "Setting this to 0 or omitting it entirely removes the limit on concurrent connections.", but we ran without this setting, and saw: WARN [NIOServerCxn.Factory:2181:NIOServerCnxn$Factory@226] - Too many connections from /10.76.251.190 - max is 10

Bug in doc possibly?

47655

No Perforce job exists for this issue.

1

32928

04/Mar/10 2:43 PM

10 years, 3 weeks, 6 days ago

Reviewed

0|i05zzb:

ZooKeeper

ZOOKEEPER-680

Including quorum config when standalone leads to crash

Bug

Open

Minor

Unresolved

Patrick D. Hunt

Vegard B. Havdal

24/Feb/10 16:51

14/Dec/19 06:06

3.2.2

3.7.0

server

0

RHEL

Include server.#=... and/or a myid file when running standalone, zk server will crash with
java.lang.NullPointerException
at org.apache.zookeeper.server.quorum.FastLeaderElection.totalOrderPredicate(FastLeaderElection.java:466)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:635)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:515)

Seen when running zk embedded in other server, using
String[] args = new String[]{zookeeperCfgFile};
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(args);

The workaround is of course to fix the config, but 3.1.1 managed to not crash on this.

70742

No Perforce job exists for this issue.

0

32929

24/Feb/10 5:23 PM

10 years, 2 weeks, 6 days ago

0|i05zzj:

ZooKeeper

ZOOKEEPER-679

Offers a node design for interacting with the Java Zookeeper client.

New Feature

Open

Major

Unresolved

Aaron Crow

24/Feb/10 13:41

05/Feb/20 07:16

3.7.0, 3.5.8

contrib, java client, tests

0

4

Following up on my conversations with Patrick and Mahadev (http://n2.nabble.com/Might-I-contribute-a-Node-design-for-the-Java-API-td4567695.html#a4567695).

This patch includes the implementation as well as unit tests. The first unit test gives a simple high level demo of using the node API.

The current implementation is simple and is only what I need withe current project I am working on. However, I am very open to any and all suggestions for improvement.

This is a proposal to support a simplified node (or File) like API into a Zookeeper tree, by wrapping the Zookeeper Java client. It is similar to Java's File API design.

Although, I'm trying to make it easier in a few spots. For example, deleting a Node recursively is done by default. I also lean toward resolving Exceptions "under the hood" when it seems appropriate. For example, if you ask a Node if it exists, and its parent doesn't even exist, you just get a false back (rather than a nasty Exception).

As for watches and ephemeral nodes, my current work does not need these things so I currently have no handling of them. But if potential users of the "Node a.k.a. File" design want these things, I'd be open to supporting them as reasonable.

70770

No Perforce job exists for this issue.

4

42174

24/Feb/10 2:22 PM

9 years, 46 weeks, 3 days ago

0|i07l13:

ZooKeeper

ZOOKEEPER-678

Browser application to view and edit the contents of a zookeeper instance

New Feature

Closed

Major

Fixed

Colin Goodheart-Smithe

22/Feb/10 15:00

27/Apr/10 12:17

08/Mar/10 20:07

3.3.0

0

3

ZOOKEEPER-418

An application which shows a tree view of the nodes currently in a zookeeper instance and allow the user to view and update the contents of the nodes as well as allowing users to add and remove nodes from the tree, similar in use to the Luke application in the Lucene project.

I have a list of other features that I want to add to this application but I wanted to gauge the response before I implemented them all. I have found this useful when debugging my application and thought that it may be useful to others.

I was going to submit this as a patch file but I have used some icon files and one library which isn't available in the maven/ivy repositories and these don't seem to work when creating a patch file using subversion. Because of this I have attached a zip containing this application to this issue. If there is a better way to submit this please let me know.

The zip contains two directories, the src directory contains the source as it would be added to the contrib folder and the build folder contains a build version of the with a runnable jar.

47656

No Perforce job exists for this issue.

4

33408

23/Feb/10 7:05 PM

9 years, 48 weeks, 2 days ago

Reviewed

0|i062xz:

ZooKeeper

ZOOKEEPER-677

c client doesn't allow ipv6 numeric connect string

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

22/Feb/10 13:20

26/Mar/10 13:25

10/Mar/10 11:34

3.2.2

3.3.0

c client

0

2

ZOOKEEPER-676

The c client doesn't handle ipv6 numeric addresses as they are colon : delmited. After splitting the host/port on : we look for the port as the second entry in the array rather than the last entry in the array.

47657

No Perforce job exists for this issue.

5

32930

23/Feb/10 1:20 AM

10 years, 3 weeks ago

Reviewed

ipv6

0|i05zzr:

ZooKeeper

ZOOKEEPER-676

50%-75% connection loss exceptions using zkpython

Bug

Open

Major

Unresolved

Unassigned

Josh Fraser

21/Feb/10 20:03

22/Feb/10 13:47

3.2.2

contrib, contrib-bindings

0

2

ZOOKEEPER-677

Mac OS X 10.5.8, MacBook Air Intel Core 2 Duo @ 1.86GHz, Python 2.5.1, ZooKeeper 3.2.2 Standalone

I get about 50-75% connection loss exceptions and about 10% Bus Error when using the contrib/zkpython zookeeper.so. Below is the exception:

2010-02-21 16:57:56,138:18481(0xb0081000):ZOO_ERROR@handle_socket_error_msg@1359: Socket [fe80::1002:885:7f00:1:2181] zk retcode=-4, errno=47(Address family not supported by protocol family): connect() call failed
Traceback (most recent call last):
File "./zksh.py", line 63, in <module>
2010-02-21 16:57:56,138:18481(0xb0081000):ZOO_INFO@check_events@1439: initiated connection to server [127.0.0.1:2181]
zkcli.dispatch(cmd,*args)
File "./zksh.py", line 56, in dispatch
returned = run(*args)
File "./zksh.py", line 48, in ls
print "\n".join(self.cmd.listNode(node))
File "/Users/josh/git/zktools/commands.py", line 22, in listNode
for path in zookeeper.get_children(self.zk, node):
zookeeper.ConnectionLossException: connection loss

I've run this in gdb and have this backtrace:

#0 free_pywatcher (pw=0x0) at src/c/zookeeper.c:199
#1 0x0025ae09 in pyzoo_exists (self=0x0, args=0x0) at src/c/zookeeper.c:765
#2 0x0018f51e in PyEval_EvalFrameEx ()
#3 0x00191173 in PyEval_EvalCodeEx ()
#4 0x0013b488 in PyFunction_SetClosure ()
#5 0x00121505 in PyObject_Call ()
#6 0x0018fcd0 in PyEval_EvalFrameEx ()
#7 0x00191173 in PyEval_EvalCodeEx ()
#8 0x0013b488 in PyFunction_SetClosure ()
#9 0x00121505 in PyObject_Call ()
#10 0x0018fcd0 in PyEval_EvalFrameEx ()
#11 0x00191173 in PyEval_EvalCodeEx ()
#12 0x0018f79d in PyEval_EvalFrameEx ()
#13 0x00191173 in PyEval_EvalCodeEx ()
#14 0x00191260 in PyEval_EvalCode ()
#15 0x001a883c in PyErr_Display ()
#16 0x001aa4ab in PyRun_InteractiveOneFlags ()
#17 0x001aa5f9 in PyRun_InteractiveLoopFlags ()
#18 0x001aaa2b in PyRun_AnyFileExFlags ()
#19 0x001b5a57 in Py_Main ()
#20 0x00001fca in ?? ()

zookeeper.c @ line 199:

void free_pywatcher( pywatcher_t *pw)
{
Py_DECREF(pw->callback);
free(pw);
}

That's as far as I've dug so far -- I ended up just writing a retry decorator to get around it for now. On the same machine, the zkCli.sh test client works flawlessly. Also, here's the Mac OS X Bus Error trace:

Process: Python [18556]
Path: /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
Identifier: Python
Version: ??? (???)
Code Type: X86 (Native)
Parent Process: bash [18436]

Interval Since Last Report: 3323078 sec
Crashes Since Last Report: 50
Per-App Interval Since Last Report: 0 sec
Per-App Crashes Since Last Report: 38

Date/Time: 2010-02-21 17:07:27.399 -0800
OS Version: Mac OS X 10.5.8 (9L31a)
Report Version: 6
Anonymous UUID: FA533BDA-50B2-47A9-931C-6F2614C741F0

Exception Type: EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000004
Crashed Thread: 0

Thread 0 Crashed:
0 zookeeper.so 0x002332bd free_pywatcher + 10 (zookeeper.c:199)
1 zookeeper.so 0x00239e09 pyzoo_exists + 984 (zookeeper.c:765)
2 org.python.python 0x0018f51e PyEval_EvalFrameEx + 17116
3 org.python.python 0x0018f700 PyEval_EvalFrameEx + 17598
4 org.python.python 0x00191173 PyEval_EvalCodeEx + 1638
5 org.python.python 0x0013b488 PyFunction_SetClosure + 2667
6 org.python.python 0x00121505 PyObject_Call + 50
7 org.python.python 0x0018fcd0 PyEval_EvalFrameEx + 19086
8 org.python.python 0x00191173 PyEval_EvalCodeEx + 1638
9 org.python.python 0x0013b488 PyFunction_SetClosure + 2667
10 org.python.python 0x00121505 PyObject_Call + 50
11 org.python.python 0x0018fcd0 PyEval_EvalFrameEx + 19086
12 org.python.python 0x00191173 PyEval_EvalCodeEx + 1638
13 org.python.python 0x00191260 PyEval_EvalCode + 87
14 org.python.python 0x001a883c PyErr_Display + 1896
15 org.python.python 0x001a8e66 PyRun_FileExFlags + 135
16 org.python.python 0x001aa7d2 PyRun_SimpleFileExFlags + 421
17 org.python.python 0x001b5a57 Py_Main + 3095
18 org.python.pythonapp 0x00001fca 0x1000 + 4042

Thread 1:
0 libSystem.B.dylib 0x9265fe0e poll$UNIX2003 + 10
1 libSystem.B.dylib 0x9262a155 _pthread_start + 321
2 libSystem.B.dylib 0x9262a012 thread_start + 34

Thread 2:
0 libSystem.B.dylib 0x9260046e __semwait_signal + 10
1 libSystem.B.dylib 0x9262adcd pthread_cond_wait$UNIX2003 + 73
2 libzookeeper_mt.2.dylib 0x00247e9f do_completion + 223
3 libSystem.B.dylib 0x9262a155 _pthread_start + 321
4 libSystem.B.dylib 0x9262a012 thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
eax: 0x00000000 ebx: 0x00239a3c ecx: 0x00000000 edx: 0x00000000
edi: 0x001efaa0 esi: 0x00000000 ebp: 0xbfffe508 esp: 0xbfffe4f0
ss: 0x0000001f efl: 0x00010286 eip: 0x002332bd cs: 0x00000017
ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037
cr2: 0x00000004

Binary Images:
0x1000 - 0x1ffe org.python.pythonapp 2.5.0 (2.5.0a0) <5aa9f0cc36fda395f965e08c96613cf5> /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
0x119000 - 0x1e5feb org.python.python 2.5 (2.5) <523ba54c654eeed6bc670db2f58a73ab> /System/Library/Frameworks/Python.framework/Versions/2.5/Python
0x232000 - 0x23ffff +zookeeper.so ??? (???) <77134e53d6dbc7bbcf783b0fc2b16d6e> /Library/Python/2.5/site-packages/zookeeper.so
0x246000 - 0x255fef +libzookeeper_mt.2.dylib ??? (???) <839ce6d5a904236d0f0112e75656ecfd> /usr/local/lib/libzookeeper_mt.2.dylib
0x8fe00000 - 0x8fe2db43 dyld 97.1 (???) <458eed38a009e5658a79579e7bc26603> /usr/lib/dyld
0x925f8000 - 0x9275fff3 libSystem.B.dylib ??? (???) <ae47ca9b1686b065f8ac4d2de09cc432> /usr/lib/libSystem.B.dylib
0x96aea000 - 0x96af1fe9 libgcc_s.1.dylib ??? (???) <e280ddf3f5fb3049e674edcb109f389a> /usr/lib/libgcc_s.1.dylib
0x96fc9000 - 0x96fcdfff libmathCommon.A.dylib ??? (???) /usr/lib/system/libmathCommon.A.dylib
0xffff0000 - 0xffff1780 libSystem.B.dylib ??? (???) /usr/lib/libSystem.B.dylib

214152

No Perforce job exists for this issue.

1

32931

21/Feb/10 11:08 PM

10 years, 5 weeks, 3 days ago

0|i05zzz:

ZooKeeper

ZOOKEEPER-675

LETest thread fails to join

Bug

Resolved

Critical

Won't Fix

Henry Robinson

Flavio Paiva Junqueira

20/Feb/10 09:32

10/Nov/11 07:30

3.5.0

leaderElection

0

After applying the patch of ZOOKEEPER-569, I observed a failure of LETest. From a cursory inspection of the log, I can tell that a leader is being elected, but some thread is not joining. At this point I'm not sure if this is a problem with the leader election implementation or the test itself.

Just to be clear, the patch of ZOOKEEPER-569 solved a real issue, but it seems that there is yet another problem with LETest.

71231

No Perforce job exists for this issue.

1

32932

09/Mar/10 1:20 PM

8 years, 20 weeks ago

0|i06007:

ZooKeeper

ZOOKEEPER-674

c client tests fail on cygwin

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

19/Feb/10 18:48

14/Dec/19 06:06

3.2.2

3.7.0

c client

0

ZOOKEEPER-586

The c client compiles on cygwin 1.5 after ZOOKEEPER-586 is applied, however not all the tests pass.

70755

No Perforce job exists for this issue.

0

32933

10 years, 5 weeks, 6 days ago

0|i0600f:

ZooKeeper

ZOOKEEPER-673

Fix observer documentation regarding leader election

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

16/Feb/10 14:03

26/Mar/10 13:25

18/Feb/10 21:08

3.3.0

0

2

We just need to remove the first two paragraphs of Section 2.

47658

No Perforce job exists for this issue.

1

32934

18/Feb/10 9:08 PM

10 years, 5 weeks, 6 days ago

Reviewed

0|i0600n:

ZooKeeper

ZOOKEEPER-672

typo nits across documentation

Improvement

Closed

Major

Fixed

Karthik K

15/Feb/10 19:39

26/Mar/10 13:25

18/Feb/10 21:34

3.2.2

3.3.0

documentation

0

1

some typo nits across the documentation. relevant forrest.xml files fixed.

47659

No Perforce job exists for this issue.

1

33409

18/Feb/10 9:11 PM

10 years, 5 weeks, 6 days ago

Reviewed

0|i062y7:

ZooKeeper

ZOOKEEPER-671

Lot of [tbd]-s ( missing links ? ) in the overview page

Improvement

Open

Major

Unresolved

Unassigned

Karthik K

15/Feb/10 15:30

13/Sep/16 14:32

documentation

1

ZOOKEEPER-2090

The zk documentation at - http://hadoop.apache.org/zookeeper/docs/r3.2.2/zookeeperOver.html , has quite a number of [tbd] in place of missing links in it.

It would be nice to see those links fixed to add value to the documentation / overview.

documentation

214151

No Perforce job exists for this issue.

0

42175

10 years, 6 weeks, 3 days ago

0|i07l1b:

ZooKeeper

ZOOKEEPER-670

zkpython leading to segfault on zookeeper server restart

Bug

Resolved

Critical

Fixed

Henry Robinson

Lei Zhang

12/Feb/10 18:21

24/Apr/14 21:44

3.2.1, 3.2.2

contrib-bindings

0

7

ZOOKEEPER-740

CentOS w/ Python 2.4

Zookeeper client using zkpython segfaults on zookeeper server restart. It is reliably reproducible using the attached script zk.py.
I'm able to stop segfault using the attached patch voyager.patch, but zkpython seems to have deeper issue on its use of watcher_dispatch - on zookeeper server restart, I see up to 6 invocation of watcher_dispatch while my script is simply sleeping in the main thread. This can't be right.

71227

No Perforce job exists for this issue.

2

32935

12/Feb/10 6:34 PM

5 years, 47 weeks, 6 days ago

0|i0600v:

ZooKeeper

ZOOKEEPER-669

watchedevent tostring should clearly output the state/type/path

Bug

Closed

Critical

Fixed

Patrick D. Hunt

12/Feb/10 15:46

26/Mar/10 13:25

23/Feb/10 13:25

3.1.2, 3.2.2

3.3.0

0

2

the current tostring method is broken

47660

No Perforce job exists for this issue.

1

32936

19/Feb/10 1:42 AM

10 years, 5 weeks, 1 day ago

Reviewed

0|i06013:

ZooKeeper

ZOOKEEPER-668

Close method in LedgerInputStream doesn't do anything

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

12/Feb/10 07:12

26/Mar/10 13:25

18/Feb/10 21:43

3.3.0

0

2

I think we should remove the close call in LedgerInputStream.

47661

No Perforce job exists for this issue.

2

32937

12/Feb/10 11:27 AM

10 years, 5 weeks, 6 days ago

0|i0601b:

ZooKeeper

ZOOKEEPER-667

java client doesn't allow ipv6 numeric connect string

Bug

Closed

Critical

Fixed

Patrick D. Hunt

11/Feb/10 14:26

20/Jun/12 06:39

12/Mar/10 13:55

3.2.2

3.3.0

java client

0

3

ZOOKEEPER-635

The java client doesn't handle ipv6 numeric addresses as they are colon (:) delmited. After splitting the host/port on : we look for the port as the second entry in the array rather than the last entry in the array.

47662

No Perforce job exists for this issue.

0

32938

11/Feb/10 5:46 PM

7 years, 40 weeks, 1 day ago

ipv6

0|i0601j:

ZooKeeper

ZOOKEEPER-666

Unsafe publication in client API

Bug

Open

Major

Unresolved

Unassigned

Martin Traverso

10/Feb/10 18:53

14/Dec/19 06:09

3.2.2

3.7.0

java client

0

2

ZOOKEEPER-837, ZOOKEEPER-835

The following code may result in a data race due to unsafe publication of a reference to "this". The call to cnxn.start() spawns threads that have access to the partially-constructed reference to the ZooKeeper object.

See http://www.ibm.com/developerworks/java/library/j-jtp0618.html for some background info.

{noformat}
public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher)
throws IOException
{
.....
cnxn = new ClientCnxn(connectString, sessionTimeout, this, watchManager);
cnxn.start();
}
{noformat}

The obvious fix is to move the call to cnxn.start() into a separate start() method.

70725

No Perforce job exists for this issue.

0

32939

11/Feb/10 1:15 AM

9 years, 17 weeks, 3 days ago

0|i0601r:

ZooKeeper

ZOOKEEPER-665

Add BookKeeper streaming documentation

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

09/Feb/10 04:40

26/Mar/10 13:25

19/Feb/10 01:26

3.3.0

contrib-bookkeeper

0

2

47663

No Perforce job exists for this issue.

2

33410

19/Feb/10 1:26 AM

10 years, 5 weeks, 6 days ago

Reviewed

0|i062yf:

ZooKeeper

ZOOKEEPER-664

BookKeeper API documentation

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

09/Feb/10 04:39

26/Mar/10 13:25

19/Feb/10 01:30

3.3.0

contrib-bookkeeper

0

2

Review and improve BookKeeper API documentation.

47664

No Perforce job exists for this issue.

1

33411

19/Feb/10 1:30 AM

10 years, 5 weeks, 6 days ago

Reviewed

0|i062yn:

ZooKeeper

ZOOKEEPER-663

hudson failure in ZKDatabaseCorruptionTest

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

02/Feb/10 12:30

26/Mar/10 13:25

08/Mar/10 20:25

3.3.0

server

0

2

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/686/

java.lang.RuntimeException: Unable to run quorum server
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:380)
at org.apache.zookeeper.test.ZkDatabaseCorruptionTest.testCorruption(ZkDatabaseCorruptionTest.java:99)
Caused by: java.io.IOException: Invalid magic number 0 != 1514884167
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:455)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:471)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:438)
at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:519)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:145)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:193)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:377)

47665

No Perforce job exists for this issue.

1

32940

05/Mar/10 2:01 PM

10 years, 3 weeks, 2 days ago

Reviewed

0|i0601z:

ZooKeeper

ZOOKEEPER-662

Too many CLOSE_WAIT socket state on a server

Bug

Closed

Major

Duplicate

Unassigned

Qian Ye

01/Feb/10 21:57

23/Nov/11 14:22

16/Jul/11 13:48

3.2.1

3.4.0

quorum

1

2

Linux 2.6.9

I have a zookeeper cluster with 5 servers, zookeeper version 3.2.1, here is the content in the configure file, zoo.cfg

======
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=5
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=2
# the directory where the snapshot is stored.
dataDir=./data/
# the port at which the clients will connect
clientPort=8181

# zookeeper cluster list
server.100=10.23.253.43:8887:8888
server.101=10.23.150.29:8887:8888
server.102=10.23.247.141:8887:8888
server.200=10.65.20.68:8887:8888
server.201=10.65.27.21:8887:8888
=====

Before the problem happened, the server.200 was the leader. Yesterday morning, I found the there were many sockets with the state of CLOSE_WAIT on the clientPort (8181), the total was over about 120. Because of these CLOSE_WAIT, the server.200 could not accept more connections from the clients. The only thing I can do under this situation is restart the server.200, at about 2010-02-01 06:06:35. The related log is attached to the issue.

67901

No Perforce job exists for this issue.

2

32941

02/Feb/10 12:29 PM

8 years, 36 weeks, 5 days ago

CLOSE_WAIT, client port

0|i06027:

ZooKeeper

ZOOKEEPER-661

Add Ruby bindings

New Feature

Open

Minor

Unresolved

Unassigned

Andrew Reynhout

30/Jan/10 13:42

03/Feb/12 12:02

contrib-bindings

0

4

MRI Ruby 1.9
JRuby 1.4

Add Ruby bindings to the ZooKeeper distribution.

Ruby presents special threading difficulties for asynchronous ZK calls (aget, watchers, etc). It looks like the simplest workaround is to patch the ZK C API.

Proposed approach will be described in comment.

Please use this ticket for discussion and suggestions.

66522

No Perforce job exists for this issue.

0

42176

01/Feb/10 12:48 PM

8 years, 7 weeks, 6 days ago

Add Ruby bindings for MRI 1.9 and JRuby 1.4 to contrib

0|i07l1j:

ZooKeeper

ZOOKEEPER-660

Clean up scripts need to change to clean up transaction logs only for a max timeout period.

Task

Open

Major

Unresolved

Mahadev Konar

27/Jan/10 18:34

14/Dec/19 06:07

3.7.0

scripts

0

ZOOKEEPER-22

As a part of ZOOKEEPER-22, the cleanup scripts will have to change to only clean up transaction logs within a max timeout period of the last transaction. This is necessary to replay any client reconnection requests that the servers will get.

70745

No Perforce job exists for this issue.

0

42177

10 years, 3 weeks, 3 days ago

0|i07l1r:

ZooKeeper

ZOOKEEPER-658

update forrest docs - AuthFLE no longer supported

Improvement

Closed

Critical

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

25/Jan/10 19:08

26/Mar/10 13:25

03/Mar/10 17:57

3.3.0

documentation, quorum, server

0

1

ZOOKEEPER-152

See ZOOKEEPER-152, the docs should be updated to reflect that AuthFLE is no longer supported. It would be a good idea to take
this opportunity to review the election docs in general and make sure they are up to date.

47666

No Perforce job exists for this issue.

1

33412

03/Mar/10 9:27 AM

10 years, 4 weeks, 1 day ago

Reviewed

0|i062yv:

ZooKeeper

ZOOKEEPER-657

Cut down the running time of ZKDatabase corruption.

Improvement

Resolved

Major

Fixed

Michi Mutsuzaki

Mahadev Konar

25/Jan/10 13:57

20/May/14 07:09

14/May/14 21:43

3.4.7, 3.5.0

tests

0

3

THe zkdatabasecorruption test takes around 180 seconds right now. It just bring down a quorum cluster up and down and corrupts some snapshots. We need to investigate why it takes that long and make it shorter so that our test run times are smaller.

70740

No Perforce job exists for this issue.

1

42178

09/May/14 6:37 PM

5 years, 44 weeks, 2 days ago

0|i07l1z:

ZooKeeper

ZOOKEEPER-656

SledgeHammer test - thread.run() deprecated

Bug

Closed

Major

Fixed

Karthik K

23/Jan/10 20:27

26/Mar/10 13:25

24/Jan/10 22:35

3.3.0

tests

0

2

Thread.run() used instead of Thread.start() .

47667

No Perforce job exists for this issue.

1

32943

23/Jan/10 8:49 PM

10 years, 9 weeks, 3 days ago

Reviewed

0|i0602n:

ZooKeeper

ZOOKEEPER-655

StringBuffer -> StringBuilder - conversion of references as necessary

Improvement

Closed

Major

Fixed

Karthik K

23/Jan/10 16:46

26/Mar/10 13:25

24/Jan/10 22:29

3.3.0

0

1

Some of the 'StringBuffer' references that do not escape the scope of the method can be better typed as StringBuilder as concurrency is not needed in that case.

47668

No Perforce job exists for this issue.

1

33413

23/Jan/10 5:29 PM

10 years, 9 weeks, 3 days ago

Reviewed

0|i062z3:

ZooKeeper

ZOOKEEPER-654

Tests should not rely on CONSOLE appender being present

Bug

Open

Minor

Unresolved

Unassigned

Henry Robinson

22/Jan/10 18:03

22/Jan/10 19:17

3.2.1

0

Tests have been failing for us in an environment where we removed the CONSOLE appender from log4j. This breaks a couple of tests in QuorumPeerMainTest at least.

I have fixed in our builds by replacing CONSOLE with ROLLINGFILE (which we are using) for the time being, but messing with the log config shouldn't break tests.

214150

No Perforce job exists for this issue.

0

32944

22/Jan/10 6:44 PM

10 years, 9 weeks, 6 days ago

0|i0602v:

ZooKeeper

ZOOKEEPER-653

hudson failure in LETest

Bug

Resolved

Major

Fixed

Patrick D. Hunt

22/Jan/10 13:07

29/Mar/12 07:40

06/Sep/11 11:57

3.5.0

quorum, server

0

2

ZOOKEEPER-569

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/675/testReport/org.apache.zookeeper.test/LETest/testLE/

junit.framework.AssertionFailedError: Threads didn't join
at org.apache.zookeeper.test.LETest.testLE(LETest.java:116)

47669

No Perforce job exists for this issue.

0

32945

22/Jan/10 2:34 PM

8 years ago

0|i06033:

ZooKeeper

ZOOKEEPER-652

package server and client separately

Improvement

Resolved

Major

Duplicate

Unassigned

Yanming Zhou

22/Jan/10 06:10

01/Nov/11 06:05

build

0

2

ZOOKEEPER-233

is it possible split zookeeper.jar to zookeeper-server.jar and zookeeper-client.jar?
so app can only use client jar

214149

No Perforce job exists for this issue.

0

33414

22/Jan/10 8:58 AM

8 years, 21 weeks, 2 days ago

0|i062zb:

ZooKeeper

ZOOKEEPER-651

Log exception trace in QuorumCnxManager.SendWorker

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

19/Jan/10 06:57

26/Mar/10 13:25

19/Jan/10 15:07

3.2.0, 3.2.1

3.3.0

0

1

There is an error log message that only outputs a constant string. It would be good to have the trace as well.

47670

No Perforce job exists for this issue.

1

33415

19/Jan/10 7:30 AM

10 years, 10 weeks ago

Reviewed

0|i062zj:

ZooKeeper

ZOOKEEPER-650

Servers cannot join in quorum

Bug

Resolved

Major

Cannot Reproduce

Unassigned

Qian Ye

19/Jan/10 05:59

06/Sep/11 11:57

3.2.1

quorum

0

2

Linux 2.6.9 x86_64

Server fails to join ensemble.

62191

No Perforce job exists for this issue.

0

32946

19/Jan/10 6:40 AM

8 years, 29 weeks, 2 days ago

Quorum

0|i0603b:

ZooKeeper

ZOOKEEPER-649

testObserver timed out once on Hudson

Bug

Resolved

Minor

Cannot Reproduce

Henry Robinson

17/Jan/10 16:37

23/Apr/14 18:13

3.3.0

3.5.0

quorum

0

1

See http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/105/testReport/org.apache.zookeeper.test/ObserverTest/testObserver/ - test has only failed once and is hard to reproduce, so am waiting for more data.

70781

No Perforce job exists for this issue.

0

32947

23/Apr/14 6:13 PM

5 years, 48 weeks, 1 day ago

0|i0603j:

ZooKeeper

ZOOKEEPER-648

Fix releaseaudit warning count to zero

Improvement

Closed

Critical

Fixed

Patrick D. Hunt

Giridharan Kesavan

17/Jan/10 13:01

26/Mar/10 13:25

21/Jan/10 17:12

3.3.0

build

0

1

ant releaseaudit -Dforrest.home=<> -Djava5.home=<>
200 unknown license should be updated with Apache License headers

releaseaudit:
[java]
[java] *****************************************************
[java] Summary
[java] -------
[java] Notes: 57
[java] Binaries: 95
[java] Archives: 16
[java] Standards: 954
[java]
[java] Apache Licensed: 610
[java] Generated Documents: 144
[java]
[java] JavaDocs are generated and so license header is optional
[java] Generated files do not required license headers
[java]
[java] 200 Unknown Licenses
[java]
[java] *******************************

47671

No Perforce job exists for this issue.

2

33416

19/Jan/10 12:54 PM

10 years, 10 weeks ago

Reviewed

0|i062zr:

ZooKeeper

ZOOKEEPER-647

hudson failure in testLeaderShutdown

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

15/Jan/10 17:33

26/Mar/10 13:25

20/Jan/10 22:08

3.3.0

server

0

2

http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/666/testReport/org.apache.zookeeper.test/QuorumTest/testLeaderShutdown/

junit.framework.AssertionFailedError: QP failed to shutdown in 30 seconds
at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:293)
at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:281)
at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:266)
at org.apache.zookeeper.test.QuorumTest.tearDown(QuorumTest.java:55)

Flavio, can you triage this one?

47672

No Perforce job exists for this issue.

2

32948

19/Jan/10 4:25 PM

10 years, 10 weeks ago

Reviewed

0|i0603r:

ZooKeeper

ZOOKEEPER-646

Namespace partitioning in ZK

New Feature

Open

Major

Unresolved

Unassigned

Karthik K

15/Jan/10 15:51

25/Oct/11 22:07

5

9

Tracking JIRA for namespace partitioning in ZK

From the mailing list (- courtesy: Mahadev / Flavio ) , discussion during Jan 2010 -

"Hi, Mahadev said it all, we have been thinking about it for a while, but
>> haven't had time to work on it. I also don't think we have a jira open for
>> it; at least I couldn't find one. But, we did put together some comments:
>>
>> http://wiki.apache.org/hadoop/ZooKeeper/PartitionedZookeeper
>>
>> One of the main issues we have observed there is that partitioning will
>> force us to change our consistency guarantees, which is far from ideal.
>> However, some users seem to be ok with it, but I'm not sure we have
>> agreement.
>>
>> In any case, please feel free to contribute or simply express your
>> interests so that we can take them into account.
>>
>> Thanks,
>> -Flavio
>>
>>
>> On Jan 15, 2010, at 12:49 AM, Mahadev Konar wrote:
>>
> >>> Hi kay,
> >>> the namespace partitioning in zookeeper has been on a back burner for a
> >>> long time. There isnt any jira open on it. There had been some
> >>> discussions
> >>> on this but no real work. Flavio/Ben have had this on there minds for a
> >>> while but no real work/proposal is out yet.
> >>>
> >>> May I know is this something you are looking for in production?
> >>>
> >>> Thanks
> >>> mahadev
"

71268

No Perforce job exists for this issue.

0

42179

08/Jun/10 1:48 PM

8 years, 22 weeks, 1 day ago

0|i07l27:

ZooKeeper

ZOOKEEPER-645

Bug in WriteLock recipe implementation?

Bug

Open

Minor

Unresolved

Mahadev Konar

Jaakko Laine

15/Jan/10 00:19

14/Dec/19 06:06

3.2.2

3.7.0

recipes

3

13

ZOOKEEPER-3221

3.2.2 java 1.6.0_12

Not sure, but there seem to be two issues in the example WriteLock:

(1) ZNodeName is sorted according to session ID first, and then according to znode sequence number. This might cause starvation as lower session IDs always get priority. WriteLock is not thread-safe in the first place, so having session ID involved in compare operation does not seem to make sense.

(2) if findPrefixInChildren finds previous ID, it should add dir in front of the ID

70799

No Perforce job exists for this issue.

3

32949

15/Jan/10 1:26 PM

1 year, 45 weeks, 3 days ago

0|i0603z:

ZooKeeper

ZOOKEEPER-644

Nightly build failed on hudson.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

14/Jan/10 13:43

26/Mar/10 13:25

15/Jan/10 17:36

3.3.0

build

0

2

the nighthly build has been failing. http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/664/. The problem seems to be
{code}
BUILD FAILED
java.lang.NoClassDefFoundError: org/apache/ivy/ant/IvyMakePom$Mapping

Total time: 15 minutes 14 seconds
{code}

47673

No Perforce job exists for this issue.

1

32950

14/Jan/10 2:54 PM

10 years, 10 weeks, 6 days ago

0|i06047:

ZooKeeper

ZOOKEEPER-643

please add support for windows os

New Feature

Resolved

Major

Duplicate

Unassigned

Yanming Zhou

14/Jan/10 04:18

14/Jan/10 12:59

0

1

add zkServer.bat for windows beside zkServer.sh and other shell

214148

No Perforce job exists for this issue.

0

33417

14/Jan/10 12:59 PM

10 years, 11 weeks ago

0|i062zz:

ZooKeeper

ZOOKEEPER-642

"exceeded deadline by N ms" floods logs

Bug

Resolved

Major

Fixed

Marc Celani

Dale Johnson

12/Jan/10 20:52

11/May/12 07:00

11/May/12 01:55

3.2.1

3.5.0

c client

2

6

virtualized linux - ec2 - ubuntu

More important zookeeper warnings are drown out by the following several times per minute:

2010-01-12 17:39:57,227:22317(0x4147eb90):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 13ms

Perhaps this is an issue with the way virtualized systems manage gettimeofday results?

Maybe the current 10ms threshold could be pushed up a bit. I notice that 95% of the messages are below 50ms.

Is there an obvious configuration change that I can make to fix this?

config file below:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/mnt/zookeeper
# the port at which the clients will connect
clientPort=2181

server.1=hbase.1:2888:3888
server.2=hbase.2:2888:3888
server.3=hbase.3:2888:3888
server.4=hbase.4:2888:3888
server.5=hbase.5:2888:3888

patch

44

No Perforce job exists for this issue.

1

32951

14/Jan/10 12:55 PM

7 years, 45 weeks, 6 days ago

0|i0604f:

ZooKeeper

ZOOKEEPER-641

Improve details about group membership recipe

Improvement

Open

Major

Unresolved

Unassigned

Adam Rosien

11/Jan/10 17:17

14/Dec/19 06:06

3.2.1

3.7.0

documentation

0

2

Regarding http://eng.kaching.com/2010/01/actually-implementing-group-management.html Patrick Hunt asked for a more complete group membership recipe from the one listed at http://hadoop.apache.org/zookeeper/docs/r3.0.0/recipes.html#sc_outOfTheBox.

The relevant text from the blog post:

One type of group management system using ZooKeeper:

* A group contains some logical service. The *meaning* of belonging to a group is typically "the instance is available for use by clients over the network".
* Services can join and leave the group. The special case of a service crashing or a network outage needs to be handled as leaving the group.
* Joined services share metadata about how to communicate with it, i.e., its IP address, base URL, etc.
* Clients can ask what instances are in the group, i.e., available.
* Clients are notified when group membership changes so they can mutate their local state.

These map onto ZooKeeper as:

* A group is a (permanent) node in the ZooKeeper hierarchy. Clients and services must be told the path to this node.
* A services joins the group by creating an ephemeral node whose parent is the group node. By using an ephemeral node, if the service dies then the service is automatically removed from the group.
* The ephemeral node's data contains the service metadata in some format like JSON, XML, Avro, Protobufs, Thrift, etc. ZooKeeper has no equivalent of HTTP's "Content-Type" header to identify the metadata representation, so services and clients must agree upon the format in some manner.
* Clients can query for the children of the group node to identify the members of the group.
* Clients can place a watch on the group node to be notified if nodes have joined or left the group.

68117

No Perforce job exists for this issue.

0

42180

11/Jan/10 5:32 PM

8 years, 37 weeks, 1 day ago

0|i07l2f:

ZooKeeper

ZOOKEEPER-640

make build.xml more configurable to ease packaging for linux distros

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Thomas Koch

08/Jan/10 09:16

26/Mar/10 13:25

04/Mar/10 15:10

3.2.1, 3.2.2

3.3.0

build

0

2

900

0%

Hi,

I started packaging Zookeeper for Debian[1][2]. Thereby I had a problem excluding contrib/rest from the build without patching the upstream tarball. Could you please add some properties to your build.xml that allow me to (de)select contribs? In the example below I can easily override the properties:

<project name="zookeepercontrib">

<property name="contribfilesetincludes" value="*/build.xml" />
<property name="contribfilesetexcludes" value="" />

<fileset id="contribfileset"
dir="."
includes="${contribfilesetincludes}"
excludes="${contribfilesetexcludes}"
/>

<target name="compile">
<subant target="jar">
<fileset refid="contribfileset" />
</subant>
</target>

Could you please also add a line to project.classpath:

<path id="project.classpath">
<fileset dir="${additional.lib.dir}" includes="*.jar"/>

For Debian I may not compile based on the jar files in lib but must use the jars already in Debian.

[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561947
[2] http://git.debian.org/?p=pkg-java/zookeeper.git

Thank you!

0%

900

47674

No Perforce job exists for this issue.

1

33418

08/Jan/10 1:28 PM

10 years, 3 weeks, 6 days ago

Reviewed

0|i06307:

ZooKeeper

ZOOKEEPER-639

Performance improvements for ZooKeeper

Improvement

Open

Major

Unresolved

Benjamin Reed

07/Jan/10 13:09

10/Oct/13 19:23

server

1

3

here are some more improvements to Zab. the patch is a bit stale, but i don't want to lose track of it. there are two big improvements:

1) transaction logs are reused. this saves time over growing the log files and if you preallocate a bunch of log files on an empty partition, you will see a nice performance boost
2) acks and commits are always sent in order, so if there is a bunch to send, they will get merged into a single ack or commit.

214147

No Perforce job exists for this issue.

1

42181

10 years, 12 weeks ago

0|i07l2n:

ZooKeeper

ZOOKEEPER-638

upgrade ivy to 2.1.0 final from 2.1.0 release candidate

Improvement

Closed

Major

Fixed

Patrick D. Hunt

07/Jan/10 12:15

26/Mar/10 13:25

15/Jan/10 17:32

3.3.0

build

0

1

2.1.0 is now available

47675

No Perforce job exists for this issue.

1

33419

08/Jan/10 11:53 AM

10 years, 10 weeks, 6 days ago

Reviewed

0|i0630f:

ZooKeeper

ZOOKEEPER-637

Trunk build is failing

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

06/Jan/10 11:26

26/Mar/10 13:33

07/Jan/10 03:42

3.3.0

build

0

2

The trunk build is failing when Hudson runs it. The problem seems to be that ivy-init is executed only once, but its definitions (in particular ivy:settings) do not persist, and the failure occurs when we run ivy-retrieve a second time, which requires the definition of ivy:settings.

It seems that the problem occur with ant 1.7.0, but not with 1.7.1, so it could be an ant issue.

47676

No Perforce job exists for this issue.

2

32952

06/Jan/10 11:42 AM

10 years, 12 weeks ago

0|i0604n:

ZooKeeper

ZOOKEEPER-636

configure.ac has instructions which override the contents of CFLAGS and CXXFLAGS.

Improvement

Closed

Major

Fixed

Maxim P. Dementiev

29/Dec/09 08:52

23/Nov/11 14:22

19/May/10 03:38

3.2.1

3.4.0

build, c client

0

The information mustn't be overridden.
The template like «CFLAGS="$CFLAGS -some-option"» should be used.

47677

No Perforce job exists for this issue.

1

33420

20/Jan/10 8:39 PM

9 years, 43 weeks, 2 days ago

Reviewed

autoconf autotools configure

0|i0630n:

ZooKeeper

ZOOKEEPER-635

Server supports listening on a specified network address

New Feature

Closed

Major

Fixed

Patrick D. Hunt

Steve Chu

22/Dec/09 03:12

22/Jul/11 19:07

09/Mar/10 00:14

3.2.1

3.3.0

server

0

2

ZOOKEEPER-1133

ZOOKEEPER-667, ZOOKEEPER-1135

The issue in maililist is located:
http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200912.mbox/%3c4ac0d28c0912210242g58230a9ds1c55361561c70d61@mail.gmail.com%3e

I have checked the server size code, seems no this option provided. This feature is useful when we have more than two network interfaces, one for Internet and others for intranet. We want to run ZooKeeper in our intranet and not be exposed to outside world.

47678

No Perforce job exists for this issue.

3

33421

05/Jan/10 3:46 PM

8 years, 35 weeks, 6 days ago

Reviewed

server option address

0|i0630v:

ZooKeeper

ZOOKEEPER-634

some elements of struct Stat have incorrect values when returned asynchronously

Bug

Resolved

Minor

Invalid

Unassigned

Andrew Reynhout

19/Dec/09 17:13

05/Jan/10 15:48

19/Dec/09 17:20

3.2.1

c client

0

OSX 10.5, 10.6 (32b and 64b, respectively)

I'm working on Ruby bindings for ZK (3.2.2) via the C API. This involves some painful kludgery due to MRI Ruby's threading model, but I believe this bug report is valid:

Some of the elements of Stat structs, when returned via the C client, have values inconsistent with the synchronous call, zkCli output, and expectations. E.g.:

{noformat:title=zkCli command line}
$ zk/bin/zkCli.sh -server localhost:2182 set /lala lala234
Connecting to localhost:2182

WATCHER::

WatchedEvent: Server state change. New state: SyncConnected
cZxid = 3
ctime = Sat Dec 19 14:32:03 EST 2009
mZxid = 69
mtime = Sat Dec 19 15:31:46 EST 2009
pZxid = 3
cversion = 0
dataVersion = 4
aclVersion = 0
ephemeralOwner = 0
dataLength = 7
numChildren = 0

{noformat}

{noformat:title=Ruby script + verbose ZK log output}
2009-12-19 15:31:48,952:45753(0x751000):ZOO_DEBUG@process_completions@1902: Switching on cptr->completion_type (async)
2009-12-19 15:31:48,952:45753(0x751000):ZOO_DEBUG@process_completions@1916: Calling COMPLETION_STAT for xid=4b2d3837 rc=0
2009-12-19 15:31:48,952:45753(0x751000):ZOO_DEBUG@process_completions@1949: --- PRE ruby callback.
-----------------------
&res.stat 0x00750e30
.czxid 3
.mzxid 69
.ctime 1261251123149
.mtime 1261254706180
.version 4
.cversion 0
.aversion 0
.ephemeralOwner 0
.dataLength 7
.numChildren 0
.pzxid 3
=======================
2009-12-19 15:31:48,952:45753(0x751000):ZOO_DEBUG@ruby_stat_completion_wrapper@1702: --- creating ruby thread.
2009-12-19 15:31:48,952:45753(0x7ee000):ZOO_DEBUG@ruby_stat_completion_wrapper_2@1685: --- invoking ruby callback.
-----------------------
cb->dc 0x002fe000
cb->rc 0
cb->ctx 0x01a03870
cb->stat 0x00750e30
->czxid 3
->mzxid 69
->ctime 1261251123149
->mtime 140735387442616
->version 0
->cversion 0
->aversion 27264392
->ephemeralOwner 4302638816
->dataLength -2100901903
->numChildren 32767
->pzxid 3
=======================
2009-12-19 15:31:48,952:45753(0x7ee000):ZOO_DEBUG@ruby_stat_completion_wrapper_2@1687: --- callback returned, freeing struct.
2009-12-19 15:31:48,952:45753(0x7ee000):ZOO_DEBUG@ruby_stat_completion_wrapper_2@1689: --- struct freed.
sync returned #<ZooKeeperFFI::Stat:0x000001013783d0>
async callback.return_code 0
attr ?? sync async
---- -- ---- -----
czxid OK 3 3
mzxid OK 69 69
ctime OK 1261251123149 1261251123149
mtime -- 1261254706180 140735387442616
version -- 4 0
cversion OK 0 0
aversion -- 0 27264392
ephemeralOwner -- 0 4302638816
dataLength -- 7 -2100901903
numChildren -- 0 32767
pzxid OK 3 3
2009-12-19 15:31:49,052:45753(0x6ce000):ZOO_DEBUG@do_io@316: IO thread terminated
2009-12-19 15:31:49,052:45753(0x751000):ZOO_DEBUG@do_completion@335: completion thread terminated
2009-12-19 15:31:49,052:45753(0x705f3be0):ZOO_INFO@zookeeper_close@2407: Closing zookeeper session 125a86bdc640020 to [127.0.0.1:2182]
{noformat}

The last set of lines, above, are output from my Ruby code, comparing the differences between sync and async calls. I added the LOG_DEBUG in zookeeper.c to dump the contents of res.stat before it gets shipped off to Ruby...as you can see, the LOG_DEBUG values match the values in the async Ruby call, but they're wrong.

I've currently only tested on OSX 10.5 (32bit) and 10.6 (64bit), but sometimes different elements are wrong on each. I can get a comprehensive list if it's useful, but as examples:

{noformat:title=data variation examples}
## aexists
# ctime (int64_t):
# 32b OK
# 64b OK
# dataLength (int32_t):
# 32b OK
# 64b NG (== -2100901903 instead of 10)

## aget
# ctime (int64_t):
# 32b OK
# 64b NG (== 0 instead of something unixtimey e.g. 1261251123149)
# dataLength (int32_t):
# 32b NG (== -1856148083 instead of 10)
# 64b OK

## aset
# ctime (int64_t):
# 32b OK
# 64b OK
# dataLength (int32_t):
# 32b OK
# 64b NG (== -2100901903 instead of 10)

{noformat}

Lastly, a snippet of my changes to zookeeper.c, hopefully showing that I haven't done any harm to the data. Note that I'm also seeing similar data variations for COMPLETION_DATA, which also returns a Stat struct. I haven't finished the Ruby code for COMPLETION_ACLLIST (again, returns a Stat), so the same issue might arise there as well.

I realize that hacking the ZK C API is not a great way to build a foreign binding. :(

{code:title=src/c/src/zookeeper.c}

case COMPLETION_STAT:
LOG_DEBUG(("Calling COMPLETION_STAT for xid=%x rc=%d",cptr->xid,rc));
if (rc) {
//cptr->c.stat_result(rc, 0, cptr->data);
ruby_stat_completion_wrapper(cptr->c.stat_result, rc, 0, (void *)cptr->data);
} else {
struct SetDataResponse res;
deserialize_SetDataResponse(ia, "reply", &res);
LOG_DEBUG(("--- PRE ruby callback.\n\
-----------------------\n\
&res.stat 0x%08x\n\
.czxid %ld\n\
.mzxid %ld\n\
.ctime %ld\n\
.mtime %ld\n\
.version %d\n\
.cversion %d\n\
.aversion %d\n\
.ephemeralOwner %ld\n\
.dataLength %d\n\
.numChildren %d\n\
.pzxid %ld\n\
=======================",
&res.stat,
res.stat.czxid, res.stat.mzxid, res.stat.ctime, res.stat.mtime,
res.stat.version, res.stat.cversion, res.stat.aversion,
res.stat.ephemeralOwner, res.stat.dataLength, res.stat.numChildren,
res.stat.pzxid ));
//cptr->c.stat_result(rc, &res.stat, cptr->data);
ruby_stat_completion_wrapper(cptr->c.stat_result, rc, &res.stat, (void *)cptr->data);
deallocate_SetDataResponse(&res);
}
break;

{code}

214146

No Perforce job exists for this issue.

0

32953

05/Jan/10 3:48 PM

10 years, 12 weeks, 2 days ago

0|i0604v:

ZooKeeper

ZOOKEEPER-633

Fetch netty using ivy for bookkeeper

Improvement

Closed

Major

Fixed

Giridharan Kesavan

Flavio Paiva Junqueira

18/Dec/09 11:30

26/Mar/10 13:25

05/Jan/10 09:05

3.3.0

contrib-bookkeeper

0

1

The patch of ZOOKEEPER-507 requires netty and to compile we now have to manually add the netty jar to {project root}/src/java/lib. GIven that we bookkeeper already fetches junit and log4j using ivy, we could also do it for netty. However, we need to configure it to fetch from a different repository.

47679

No Perforce job exists for this issue.

4

33422

04/Jan/10 8:02 AM

10 years, 12 weeks, 1 day ago

0|i06313:

ZooKeeper

ZOOKEEPER-632

add some documentation or FAQ on how to do rolling upgrade

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

16/Dec/09 19:21

26/Mar/10 13:25

21/Jan/10 13:31

3.3.0

documentation

0

2

add some detail on how to do a rolling upgrade. the benefits, etc...

47680

No Perforce job exists for this issue.

0

33423

10 years, 10 weeks ago

0|i0631b:

ZooKeeper

ZOOKEEPER-631

zkpython's C code could do with a style clean-up

Improvement

Closed

Minor

Fixed

Henry Robinson

16/Dec/09 18:08

23/Nov/11 14:22

22/Apr/10 02:10

3.3.1, 3.4.0

contrib-bindings

0

1

ZOOKEEPER-740, ZOOKEEPER-742

Inconsistent formatting / use of parenthesis / some error checking - all need fixing.

Also, the documentation in the header file could do with a reformat.

47681

No Perforce job exists for this issue.

1

33424

18/Apr/10 11:48 PM

9 years, 49 weeks ago

Reviewed

0|i0631j:

ZooKeeper

ZOOKEEPER-630

Trunk has duplicate ObserverTest.java files

Bug

Closed

Major

Fixed

Henry Robinson

16/Dec/09 02:39

26/Mar/10 13:25

16/Dec/09 13:30

3.3.0

tests

0

2

There are two identical ObserverTest.java files in trunk.

47682

No Perforce job exists for this issue.

1

32954

16/Dec/09 5:39 AM

10 years, 15 weeks ago

Reviewed

0|i06053:

ZooKeeper

ZOOKEEPER-629

FLELostMessageTest assumes that the first zxid on a startup of quorum is -1.

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Mahadev Konar

16/Dec/09 00:00

01/May/13 22:29

17/Dec/09 21:20

3.3.0

0

2

ZOOKEEPER-596

FLELostMessageTest assumes that the first zxid exchange will be -1 zxid. WIth ZOOKEEPER-596 the zxid would be 0 and not -1. So the corresponding change needs to be made to this test.

47683

No Perforce job exists for this issue.

1

32955

16/Dec/09 8:03 AM

10 years, 14 weeks, 6 days ago

Reviewed

0|i0605b:

ZooKeeper

ZOOKEEPER-628

the ephemeral node wouldn't disapper due to session close error

Bug

Resolved

Major

Cannot Reproduce

Unassigned

Qian Ye

15/Dec/09 20:58

06/Sep/11 11:58

3.2.1

server

0

1

Linux 2.6.9 x86_64

I find a very strange scenario today, I'm not sure how it happen, I just found it like this. Maybe you can give me some information about it, my Zookeeper Server is version 3.2.1.

My Zookeeper cluster contains three servers, with ip: 10.81.12.144,10.81.12.145,10.81.12.141. I wrote a client to create ephemeral node under znode: se/diserver_tc. The client runs on the server with ip 10.81.13.173. The client can create a ephemeral node on zookeeper server and write the host ip (10.81.13.173) in to the node as its data. There is only one client process can be running at a time, because the client will listen to a certain port.

It is strange that I found there were two ephemeral node with the ip 10.81.13.173 under znode se/diserver_tc.
se/diserver_tc/diserver_tc0000000067
STAT:
czxid: 124554079820
mzxid: 124554079820
ctime: 1260609598547
mtime: 1260609598547
version: 0
cversion: 0
aversion: 0
ephemeralOwner: 226627854640480810
dataLength: 92
numChildren: 0
pzxid: 124554079820

se/diserver_tc/diserver_tc0000000095
STAT:
czxid: 128849019107
mzxid: 128849019107
ctime: 1260772197356
mtime: 1260772197356
version: 0
cversion: 0
aversion: 0
ephemeralOwner: 154673159808876591
dataLength: 92
numChildren: 0
pzxid: 128849019107

There are TWO with different session id! And after I kill the client process on the server 10.81.13.173, the se/diserver_tc/diserver_tc0000000095 node disappear, but the se/diserver_tc/diserver_tc0000000067 stay the same. That means it is not my coding mistake to create the node twice. I checked several times and I'm sure that there is no another client instance running. And I use the 'stat' command to check the three zookeeper servers, and there is no client from 10.81.13.173,

$echo stat | nc 10.81.12.144 2181
Zookeeper version: 3.2.1-808558, built on 08/27/2009 18:48 GMT
Clients:
/10.81.13.173:35676[1](queued=0,recved=0,sent=0) # it is caused by the nc process

Latency min/avg/max: 0/3/254
Received: 11081
Sent: 0
Outstanding: 0
Zxid: 0x1e000001f5
Mode: follower
Node count: 32

$ echo stat | nc 10.81.12.141 2181
Zookeeper version: 3.2.1-808558, built on 08/27/2009 18:48 GMT
Clients:
/10.81.12.152:58110[1](queued=0,recved=10374,sent=0)
/10.81.13.173:35677[1](queued=0,recved=0,sent=0) # it is caused by the nc process

Latency min/avg/max: 0/0/37
Received: 37128
Sent: 0
Outstanding: 0
Zxid: 0x1e000001f5
Mode: follower
Node count: 26

$ echo stat | nc 10.81.12.145 2181
Zookeeper version: 3.2.1-808558, built on 08/27/2009 18:48 GMT
Clients:
/10.81.12.153:19130[1](queued=0,recved=10624,sent=0)
/10.81.13.173:35678[1](queued=0,recved=0,sent=0) # it is caused by the nc process

Latency min/avg/max: 0/2/213
Received: 26700
Sent: 0
Outstanding: 0
Zxid: 0x1e000001f5
Mode: leader
Node count: 26

The three 'stat' commands show different Node count!

62190

No Perforce job exists for this issue.

0

32956

16/Dec/09 2:27 PM

8 years, 29 weeks, 2 days ago

session, ephemeral node, server

0|i0605j:

ZooKeeper

ZOOKEEPER-627

zkpython arbitrarily restricts the size of a 'get' to 512 bytes

Bug

Closed

Major

Fixed

Henry Robinson

15/Dec/09 19:45

26/Mar/10 13:25

16/Dec/09 18:28

3.3.0

contrib-bindings

0

3

Reported on the list:

"
I'm working on using ZooKeeper for an internal application at Digg. I've been using the zkpython package and I just noticed that the data I was receiving from a zookeeper.get() call was being truncated. After some quick digging I found that zookeeper.c limits the data returned to 512 characters (see http://svn.apache.org/viewvc/hadoop/zookeeper/tags/release-3.2.2/src/contrib/zkpython/src/c/zookeeper.c?view=markup line 855).

Is there a reason for this? The only information regarding node size that I've read is that it should not exceed 1MB so this limit seems a bit arbitrary and restrictive.

Thanks for the great work!

Rich"

47684

No Perforce job exists for this issue.

3

32957

16/Dec/09 1:12 AM

10 years, 15 weeks ago

fix to remove restrictions on get in the zkpython library.

Reviewed

0|i0605r:

ZooKeeper

ZOOKEEPER-626

ensure the c/java cli's print xid/sessionid/etc... in hex

Bug

Closed

Major

Fixed

Patrick D. Hunt

15/Dec/09 12:54

26/Mar/10 13:25

21/Jan/10 19:49

3.2.1

3.3.0

c client, java client

0

2

Java/c clients should output xid/sessionids (incl ephemeralowner) in hex format

private static void printStat(Stat stat) {
System.err.println("cZxid = " + stat.getCzxid());
System.err.println("ctime = " + new Date(stat.getCtime()).toString());
System.err.println("mZxid = " + stat.getMzxid());
System.err.println("mtime = " + new Date(stat.getMtime()).toString());
System.err.println("pZxid = " + stat.getPzxid());
System.err.println("cversion = " + stat.getCversion());
System.err.println("dataVersion = " + stat.getVersion());
System.err.println("aclVersion = " + stat.getAversion());
System.err.println("ephemeralOwner = " + stat.getEphemeralOwner());
System.err.println("dataLength = " + stat.getDataLength());
System.err.println("numChildren = " + stat.getNumChildren());
}

47685

No Perforce job exists for this issue.

1

32958

20/Jan/10 3:49 PM

10 years, 9 weeks, 5 days ago

Reviewed

0|i0605z:

ZooKeeper

ZOOKEEPER-625

configure/change build scripts to generate cppunit reports

Improvement

Open

Major

Unresolved

Unassigned

Giridharan Kesavan

15/Dec/09 11:00

build

0

1

214145

No Perforce job exists for this issue.

0

42182

10 years, 15 weeks, 2 days ago

0|i07l2v:

ZooKeeper

ZOOKEEPER-624

The C Client cause core dump when receive error data from Zookeeper Server

Bug

Closed

Major

Fixed

Mahadev Konar

Qian Ye

14/Dec/09 22:17

26/Mar/10 13:25

16/Mar/10 20:05

3.2.0

3.3.0

c client

0

3

Linux 2.6.9 x86_64

I encountered a problem today that the Zookeeper C Client (version 3.2.0) core dump when reconnected and did some operations on the zookeeper server which just restarted. The gdb infomation is like:

(gdb) bt
#0 0x000000302af71900 in memcpy () from /lib64/tls/libc.so.6
#1 0x000000000047bfe4 in ia_deserialize_string (ia=Variable "ia" is not available.) at src/recordio.c:270
#2 0x000000000047ed20 in deserialize_CreateResponse (in=0x9cd870, tag=0x50a74e "reply", v=0x409ffe70) at generated/zookeeper.jute.c:679
#3 0x000000000047a1d0 in zookeeper_process (zh=0x9c8c70, events=Variable "events" is not available.) at src/zookeeper.c:1895
#4 0x00000000004815e6 in do_io (v=Variable "v" is not available.) at src/mt_adaptor.c:310
#5 0x000000302b80610a in start_thread () from /lib64/tls/libpthread.so.0
#6 0x000000302afc6003 in clone () from /lib64/tls/libc.so.6
#7 0x0000000000000000 in ?? ()
(gdb) f 1
#1 0x000000000047bfe4 in ia_deserialize_string (ia=Variable "ia" is not available.) at src/recordio.c:270
270 in src/recordio.c
(gdb) info locals
priv = (struct buff_struct *) 0x9cd8d0
len = -1
rc = Variable "rc" is not available.

According to the source code,
int ia_deserialize_string(struct iarchive *ia, const char *name, char **s)
{
struct buff_struct *priv = ia->priv;
int32_t len;
int rc = ia_deserialize_int(ia, "len", &len);
if (rc < 0)
return rc;
if ((priv->len - priv->off) < len) {
return -E2BIG;
}
*s = malloc(len+1);
if (!*s) {
return -ENOMEM;
}
memcpy(*s, priv->buffer+priv->off, len);
(*s)[len] = '\0';
priv->off += len;
return 0;
}

the variable len is set by ia_deserialize_int, and the returned len doesn't been checked, so the client segment fault when trying to memcpy -1 byte data.
In the source file recordio.c, there are many functions which don't check the returned len. They all might cause segment fault in some kind of situations.

47686

No Perforce job exists for this issue.

2

32959

14/Dec/09 10:39 PM

10 years, 2 weeks, 2 days ago

Reviewed

c client, core dump

0|i06067:

ZooKeeper

ZOOKEEPER-623

ClientBase in bookkeeper.util requires junit

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

14/Dec/09 03:49

01/May/13 22:29

12/Jan/10 15:12

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-534

Class org.apache.bookkeeper.util.ClientBase requires junit, and when I tried to just compile bookkeeper, no test, with the patch of ZOOKEEPER-534, compilation failed.

47687

No Perforce job exists for this issue.

4

32960

14/Dec/09 6:42 AM

10 years, 11 weeks, 2 days ago

Reviewed

0|i0606f:

ZooKeeper

ZOOKEEPER-622

Test for pending watches in send_set_watches should be moved

Bug

Closed

Major

Fixed

Benjamin Reed

Steven Cheng

12/Dec/09 17:22

26/Mar/10 13:25

06/Mar/10 00:35

3.3.0

c client

0

2

ZOOKEEPER-685

Valgrind found:

{quote}
==2357== Conditional jump or move depends on uninitialised value(s)
==2357== at 0x807FDCA: check_events (zookeeper.c:1180)
==2357== by 0x808043A: zookeeper_process (zookeeper.c:1775)
==2357== by 0x806A21B: Zookeeper_close::testCloseConnected1() (TestZookeeperClose.cc:161)
==2357== by 0x806C6BF: CppUnit::TestCaller<Zookeeper_close>::runTest() (TestCaller.h:166)
{quote}

zookeeper.c:1180 was the first if in send_set_watches.

47688

No Perforce job exists for this issue.

4

32961

12/Dec/09 8:05 PM

10 years, 3 weeks, 5 days ago

Reviewed

0|i0606n:

ZooKeeper

ZOOKEEPER-621

hudson failure ZooKeeper-trunk/621 - clover issue

Bug

Closed

Critical

Fixed

Giridharan Kesavan

Patrick D. Hunt

12/Dec/09 16:36

26/Mar/10 13:25

14/Dec/09 05:42

3.3.0

build

0

2

Giri can you take a look? Any recent changes on hudson that could have caused this?

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/621/

jute:
[javac] Compiling 38 source files to /grid/0/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/build/classes
[clover] Clover Version 2.4.3, built on March 09 2009 (build-756)
[clover] Loaded from: /homes/hudson/tools/clover/latest/lib/clover.jar
[clover] Clover: Open Source License registered to Apache.
[clover] Failed to create temp directory
[clover] ** Error(s) occurred and the instrumentation process can't continue.

BUILD FAILED
/grid/0/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/build.xml:879: The following error occurred while executing this line:
/grid/0/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/build.xml:199: com.cenqua.clover.CloverException: Failed to create temp directory

47689

No Perforce job exists for this issue.

0

32962

14/Dec/09 5:42 AM

10 years, 15 weeks, 3 days ago

0|i0606v:

ZooKeeper

ZOOKEEPER-620

hudson is not reporting compiler warning correctly

Bug

Closed

Major

Fixed

Giridharan Kesavan

Patrick D. Hunt

10/Dec/09 16:34

26/Mar/10 13:25

19/Jan/10 05:33

3.3.0

tests

0

2

http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/590/warningsResult/HIGH/

If you click on any of these links you will see that these are not compiler warnings:
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/590/warningsResult/HIGH/file.-1602148846/

Giri can you take a look at these and resolve?

47690

No Perforce job exists for this issue.

0

32963

15/Jan/10 5:29 PM

10 years, 10 weeks, 2 days ago

0|i06073:

ZooKeeper

ZOOKEEPER-619

hudson test failure in QuorumTest -- timeout error

Bug

Closed

Critical

Invalid

Patrick D. Hunt

10/Dec/09 16:31

26/Mar/10 13:25

11/Dec/09 14:09

3.3.0

0

1

ZOOKEEPER-618

Hudson is not reporting this correctly but:
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/612/

[junit] 2009-12-03 08:46:27,297 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:11301:LeaderElection@109] - 5 -> 2
[junit] Running org.apache.zookeeper.test.QuorumTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Test org.apache.zookeeper.test.QuorumTest FAILED (timeout)
[junit] 2009-12-03 08:46:28,390 - INFO [main:PortAssignment@31] - assigning port 11221
[junit] 2009-12-03 08:46:28,393 - INFO [main:PortAssignment@31] - assigning port 11222
[junit] Running org.apache.zookeeper.test.QuorumZxidSyncTest

but this makes no sense - how is this a timeout?

One concern is - perhaps this is a deadlock?

214144

No Perforce job exists for this issue.

0

32964

10 years, 15 weeks, 6 days ago

0|i0607b:

ZooKeeper

ZOOKEEPER-618

hudson not reporting failures correctly

Bug

Resolved

Critical

Invalid

Giridharan Kesavan

Patrick D. Hunt

10/Dec/09 16:29

15/Dec/09 11:06

3.3.0

tests

0

ZOOKEEPER-619

See
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/612/
this test failed however there is no indication in the hudson page why it failed.

Looking at the raw console output I see

[junit] Running org.apache.zookeeper.test.QuorumTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Test org.apache.zookeeper.test.QuorumTest FAILED (timeout)

So it seems there is an error in hudson reporting.

Giri can you look into this?

214143

No Perforce job exists for this issue.

0

32965

11/Dec/09 1:17 PM

10 years, 15 weeks, 2 days ago

0|i0607j:

ZooKeeper

ZOOKEEPER-617

improve cluster setup documentation in forrest

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Dec/09 14:11

26/Mar/10 13:25

12/Mar/10 13:27

3.3.0

documentation

0

2

ZOOKEEPER-601

http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_zkMulitServerSetup

1) the config file is missing line returns
2) call out setting up the myid file as it's own bullet, otw it's too easy to miss
3) we should make sure the values we use in examples are consistent, and resonable defaults

47691

No Perforce job exists for this issue.

0

32966

12/Mar/10 1:27 PM

10 years, 2 weeks, 6 days ago

0|i0607r:

ZooKeeper

ZOOKEEPER-616

Provide a function to parse out the name and the sequence number from a zknode path

New Feature

Open

Minor

Unresolved

Thomas Koch

Avery Ching

09/Dec/09 17:32

14/Dec/19 06:08

3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.2.0, 3.2.1

3.7.0

c client, java client

0

ZOOKEEPER-849

Given a zookeeper path and knowing it was created with the SEQUENCE flag, it would be nice to be able to get the sequence number and the name. Currently, it is not documented how many bytes the sequence number uses in the path (Mahadev told me 10 for 3.1.1 for example), and having a function to retrieve this data would hide the actual number of bytes used and provide the useful functionality for users.

70790

No Perforce job exists for this issue.

0

42183

11/Dec/09 2:20 PM

9 years, 17 weeks, 3 days ago

0|i07l33:

ZooKeeper

ZOOKEEPER-615

wrong javadoc for create with a sequence flag

Bug

Closed

Major

Fixed

Mahadev Konar

09/Dec/09 17:17

26/Mar/10 13:25

11/Dec/09 14:39

3.3.0

0

2

The javadoc for create with a sequence flag mentions a suffix of "\_i" but the true suffix is just "i" and no "\_".

47692

No Perforce job exists for this issue.

1

32967

10/Dec/09 12:41 PM

10 years, 15 weeks, 3 days ago

Reviewed

0|i0607z:

ZooKeeper

ZOOKEEPER-614

Improper synchronisation in getClientCnxnCount

Bug

Closed

Major

Fixed

Henry Robinson

07/Dec/09 18:31

26/Mar/10 13:25

11/Dec/09 18:04

3.2.1, 3.2.2

3.3.0

server

0

3

getClientCnxnCount reads from Factory.ipMap without synchronizing.

47693

No Perforce job exists for this issue.

1

32968

11/Dec/09 1:42 PM

10 years, 15 weeks, 3 days ago

Reviewed

0|i06087:

ZooKeeper

ZOOKEEPER-613

add ganglia monitoring support as contrib

Improvement

Resolved

Major

Duplicate

Unassigned

Patrick D. Hunt

07/Dec/09 14:28

05/Jul/10 14:27

contrib, scripts

0

1

ZOOKEEPER-744

It would be great to add ganglia monitoring support for ZooKeeper Cluster. AFAIK this means writing a ganglia plugin that knows
how to talk to ZK servers (either JMX, or the command port via 4letterwords)

214142

No Perforce job exists for this issue.

0

33425

02/May/10 12:35 PM

9 years, 38 weeks, 3 days ago

0|i0631r:

ZooKeeper

ZOOKEEPER-612

Make Zookeeper C client can be compiled by gcc of early version

Improvement

Closed

Major

Fixed

Qian Ye

07/Dec/09 03:59

26/Mar/10 13:25

25/Jan/10 20:07

3.2.1

3.3.0

c client

0

1

Linux

The original C Client, Version 3.2.1, cannot be compiled successfully by the gcc of early version, due some declaration restriction. To compile the source code on the server with gcc of early version, I made some modification on the original source. What's more, some extra codes are added to make the client be compatible with the hosts list format: ip1:port1, ip2:port2... There is often a space after this kind of comma.

47694

No Perforce job exists for this issue.

6

33426

05/Jan/10 12:45 PM

10 years, 9 weeks, 1 day ago

to be able to compile c library with gcc 2.96 and greater.

Reviewed

gcc, c client

0|i0631z:

ZooKeeper

ZOOKEEPER-611

hudson build failiure

Bug

Closed

Major

Fixed

Mahadev Konar

06/Dec/09 20:47

26/Mar/10 13:25

08/Dec/09 15:21

3.1.1, 3.2.1

3.1.2, 3.2.2, 3.3.0

0

2

The hudson build failure failed again on

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/582/

47695

No Perforce job exists for this issue.

5

32969

06/Dec/09 10:19 PM

10 years, 16 weeks, 2 days ago

Reviewed

0|i0608f:

ZooKeeper

ZOOKEEPER-610

cleanup final fields, esp those used for locking

Bug

Closed

Minor

Fixed

Patrick D. Hunt

04/Dec/09 01:50

26/Mar/10 13:25

11/Dec/09 17:57

3.3.0

java client, server

0

2

There are a number of places where we have non-final fields that could (should) be declared as final.

47696

No Perforce job exists for this issue.

1

32970

11/Dec/09 1:59 PM

10 years, 15 weeks, 6 days ago

Reviewed

0|i0608n:

ZooKeeper

ZOOKEEPER-609

ObserverTest failure "zk should not be connected expected not same"

Bug

Closed

Blocker

Fixed

Henry Robinson

Patrick D. Hunt

03/Dec/09 14:32

26/Mar/10 13:25

16/Dec/09 13:29

3.3.0

quorum, server

0

2

ObserverTest failed running on 8core

I ran the test as:

ant -Dtest.junit.output.format=xml -Dtest.output -Dtestcase=AsyncHammerTest clean test-core-java &> test.out

47697

No Perforce job exists for this issue.

2

32971

03/Dec/09 5:54 PM

10 years, 15 weeks ago

Reviewed

0|i0608v:

ZooKeeper

ZOOKEEPER-608

Receipt of ACK from observer should not be logged as ERROR

Improvement

Closed

Critical

Fixed

Henry Robinson

02/Dec/09 19:26

26/Mar/10 13:25

20/Jan/10 20:45

3.3.0

0

2

Observers - in general - don't send ACKs. But a couple of times they need to. Currently, these are all logged as an ERROR, which is wrong. They should at most be WARN (and this would probably be confusing to the user). INFO might be better.

47698

No Perforce job exists for this issue.

1

33427

15/Jan/10 5:27 PM

10 years, 10 weeks ago

Reviewed

0|i06327:

ZooKeeper

ZOOKEEPER-607

improve bookkeeper overview

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

Benjamin Reed

02/Dec/09 11:22

26/Mar/10 13:25

19/Feb/10 02:02

3.3.0

contrib-bookkeeper

0

1

ZOOKEEPER-507

fix the overview section in the bookkeeper documentation to introduce the programmer/admin to bookkeeper before giving the details.

47699

No Perforce job exists for this issue.

6

33428

02/Feb/10 11:16 AM

10 years, 5 weeks, 6 days ago

Reviewed

0|i0632f:

ZooKeeper

ZOOKEEPER-606

bin scripts don't work in cygwin (spaces in paths)

Bug

Closed

Major

Fixed

Patrick D. Hunt

01/Dec/09 20:10

26/Mar/10 13:25

17/Dec/09 17:09

3.2.1

3.3.0

scripts

0

2

ZOOKEEPER-426

the scripts in bin fail under cygwin due to spaces not handled properly

47700

No Perforce job exists for this issue.

2

32972

14/Dec/09 8:54 PM

10 years, 14 weeks, 6 days ago

Committed revision 891944.

Reviewed

0|i06093:

ZooKeeper

ZOOKEEPER-605

remove datatreebuilder classes from the codebase.

Improvement

Open

Major

Unresolved

Mahadev Konar

01/Dec/09 17:57

14/Dec/19 06:06

3.7.0

0

the current trunk has datatreebuilder classes which adds an unnecessary layer of abastraction without any need for it. It just causes confusion and unreadable code. We should get rid of these classes.

70761

No Perforce job exists for this issue.

0

42184

10 years, 3 weeks, 3 days ago

0|i07l3b:

ZooKeeper

ZOOKEEPER-604

zk needs to prevent export of any symbol not listed in their api

Bug

Closed

Critical

Fixed

Mahadev Konar

Alex Newman

01/Dec/09 16:01

03/Sep/10 13:21

12/Mar/10 16:44

3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.2.1, 3.2.2, 3.3.0, 4.0.0

3.3.0

c client

0

4

ZOOKEEPER-295

All

Currently the zookeeper seems to be exporting symbols not in the api. An example of this seems to be the symbol hash, which interferes with me using memcached and zookeeper in the same program.

47701

No Perforce job exists for this issue.

1

32973

01/Dec/09 4:28 PM

10 years, 2 weeks, 3 days ago

Reviewed

0|i0609b:

ZooKeeper

ZOOKEEPER-603

zkpython should do a better job of freeing memory under error conditions

Bug

Closed

Major

Duplicate

Lei Zhang

Henry Robinson

01/Dec/09 14:27

23/Nov/11 14:22

12/Aug/10 18:29

3.2.1

3.4.0

contrib-bindings

0

1

The general pattern is that the construction of a collection might fail, but the module is not freeing the memory that it has already allocated. Exceptions that are raised during this process aren't always propagated back to the Python side either.

214141

No Perforce job exists for this issue.

0

32974

01/Dec/09 3:32 PM

10 years, 3 weeks, 1 day ago

0|i0609j:

ZooKeeper

ZOOKEEPER-602

log all exceptions not caught by ZK threads

Bug

Resolved

Blocker

Fixed

Rakesh Radhakrishnan

Patrick D. Hunt

01/Dec/09 14:25

22/Jun/15 10:11

18/Jun/15 15:00

3.2.1

3.4.7, 3.5.0

java client, server

0

17

ZOOKEEPER-2029, ZOOKEEPER-597, ZOOKEEPER-1907

the java code should add a ThreadGroup exception handler that logs at ERROR level any uncaught exceptions thrown by Thread run methods.

39764

No Perforce job exists for this issue.

8

32975

04/Dec/09 2:29 PM

4 years, 39 weeks, 3 days ago

0|i0609r:

ZooKeeper

ZOOKEEPER-601

allow configuration of session timeout min/max bounds

Improvement

Closed

Major

Fixed

Patrick D. Hunt

30/Nov/09 18:12

26/Mar/10 13:25

12/Mar/10 13:27

3.2.1

3.3.0

server

0

1

ZOOKEEPER-617, ZOOKEEPER-595

ZK servers currently enforce a min/max boundary on client session timeout relative to the ticktime setting, detailed here:
http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions

In general there are good reasons for this however in some cases, in particular with HBase region servers, we have seen a need to allow this bound to be set differently (higher). The Sun jvm can GC pause for very long times (in some cases we've seen 4 minutes even with the "realtime" gc.

It would be good to allow this bound to be set via configuration parameters.

Note: 4letterword and JMX integration would be needed.

47702

No Perforce job exists for this issue.

2

33429

19/Feb/10 7:46 PM

10 years, 2 weeks, 3 days ago

Reviewed

0|i0632n:

ZooKeeper

ZOOKEEPER-600

TODO pondering about allocation behavior in zkpython may be removed

Bug

Closed

Trivial

Fixed

Gustavo Niemeyer

30/Nov/09 17:22

26/Mar/10 13:25

17/Dec/09 18:39

3.2.1

3.3.0

contrib-bindings

0

3

I suppose the TODO below is referring to the "path" variable which is passed in as an output variable to PyArg_ParseTuple right below. The TODO may be removed, since the code is right. Code using PyArg_ParseTuple will borrow the reference from the calling code, since there's a stack behind the call to the enclosing function (pyzoo_get_children in this case) which won't go away until the function returns.

Index: src/contrib/zkpython/src/c/zookeeper.c
===================================================================
--- src/contrib/zkpython/src/c/zookeeper.c (revision 885582)
+++ src/contrib/zkpython/src/c/zookeeper.c (working copy)
@@ -774,8 +774,6 @@

static PyObject *pyzoo_get_children(PyObject *self, PyObject *args)
{
- // TO DO: Does Python copy the string or the reference? If it's the former
- // we should free the String_vector
int zkhid;
char *path;
PyObject *watcherfn = Py_None;

47703

No Perforce job exists for this issue.

1

32976

30/Nov/09 5:33 PM

10 years, 14 weeks, 6 days ago

Reviewed

0|i0609z:

ZooKeeper

ZOOKEEPER-599

Changes to FLE and QuorumCnxManager to support Observers

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

26/Nov/09 11:55

26/Mar/10 13:25

15/Dec/09 17:24

3.3.0

0

3

ZOOKEEPER-506

Observers currently can only use electionAlg 0, which is not the default, supported leader implementation. This issue will fix it.

47704

No Perforce job exists for this issue.

4

33430

28/Nov/09 9:37 AM

10 years, 15 weeks, 1 day ago

Reviewed

0|i0632v:

ZooKeeper

ZOOKEEPER-598

LearnerHandler is misspelt in the thread's constructor

Bug

Closed

Trivial

Fixed

Henry Robinson

25/Nov/09 14:53

26/Mar/10 13:28

30/Nov/09 07:04

3.2.2

quorum

0

3

Typo in thread constructor. Oops.

47705

No Perforce job exists for this issue.

1

32977

30/Nov/09 1:39 AM

10 years, 17 weeks, 1 day ago

0|i060a7:

ZooKeeper

ZOOKEEPER-597

ASyncHammerTest is failing intermittently on hudson trunk

Bug

Closed

Critical

Fixed

Benjamin Reed

Patrick D. Hunt

25/Nov/09 13:18

26/Mar/10 13:25

08/Dec/09 14:15

3.1.1, 3.2.1

3.1.2, 3.2.2, 3.3.0

tests

0

2

ZOOKEEPER-602

ASyncHammerTest is failing intermittently on hudson trunk. There is no clear reason why this is happening, but
it seems from the logs that a session connection to a follower is failing during session establishment - the
failure seems to be a problem either on the follower or leader. The server gets the session create request, but
it stalls in the request processor pipeline. (we see it go in, but we do not see it com eout)

unfortunately all efforts to reproduce this on non-hudson trunk have failed. Even trying to reproduce by
running on hudson host itself (manually) has failed.

We need to instrument the client session creation code in the test to dump the thread stack if the
session creation fails.

47706

No Perforce job exists for this issue.

9

32978

25/Nov/09 2:06 PM

10 years, 16 weeks, 2 days ago

Reviewed

0|i060af:

ZooKeeper

ZOOKEEPER-596

The last logged zxid calculated by zookeeper servers could cause problems in leader election if data gets corrupted.

Bug

Closed

Major

Fixed

Mahadev Konar

24/Nov/09 17:51

01/May/13 22:29

17/Dec/09 21:20

3.2.1

3.3.0

0

2

ZOOKEEPER-629

It is possible that the last loggged zxid as reported by all the servers during leader election is not the last zxid that the server can upload data to. It is very much possible that some transaction or snapshot gets corrupted and the servers actually do not have valid data till last logged zxid. We need to make sure that what the servers report as there last logged zxid, they are able to load data till that zxid.

47707

No Perforce job exists for this issue.

6

32979

16/Dec/09 7:52 AM

10 years, 14 weeks, 6 days ago

Reviewed

0|i060an:

ZooKeeper

ZOOKEEPER-595

A means of asking quorum what conifguration it is running with

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Michael Stack

24/Nov/09 16:45

26/Mar/10 13:25

12/Mar/10 13:28

3.3.0

jmx, server

0

2

ZOOKEEPER-601

I'd like to ask a running quorum what its configuration is. I'd want to know stuff like session timeout and tick times.

Use case is that in hbase there is no zoo.cfg usually; the configuration is manufactured and piped to the starting zk server. I want to know if all of the manufactured config. 'took' or how zk interpreted it.

47708

No Perforce job exists for this issue.

0

33431

24/Nov/09 5:06 PM

10 years, 2 weeks, 6 days ago

0|i06333:

ZooKeeper

ZOOKEEPER-594

Adding documentation to zkpython to help out with the setup for finding libzookeeper_mt.so.2

Improvement

Open

Minor

Unresolved

Erik Holstad

24/Nov/09 12:25

24/Nov/09 18:27

contrib

0

Have been tested on Fedora 9

When setting up and using the zkpython binding, sometimes you encounter the problem that python cannot find the file "libzookeeper_mt.so.2". This can be fixed in most cases by for example adding the line "export LD_LIBRARY_PATH=/usr/local/lib/" to your .bashrc file. I some cases though, when for example running your python script from within a fcgi context and a lighttpd server it still isn't able to find the file.

214140

No Perforce job exists for this issue.

2

42185

24/Nov/09 2:09 PM

10 years, 18 weeks, 2 days ago

0|i07l3j:

ZooKeeper

ZOOKEEPER-593

java client api does not allow client to access negotiated session timeout

Bug

Closed

Major

Fixed

Patrick D. Hunt

24/Nov/09 11:54

26/Mar/10 13:25

25/Jan/10 21:19

3.3.0

java client

0

2

ZOOKEEPER-587

The java client api does not allow the client to access the negotiated session timeout (c does allow this).

In some cases the client may not get the requested timeout (server applies a min/max bound) in which case
the client user code may want to examine the timeout it did receive.

47709

No Perforce job exists for this issue.

1

32980

23/Jan/10 3:39 AM

10 years, 9 weeks, 1 day ago

Reviewed

0|i060av:

ZooKeeper

ZOOKEEPER-592

BookKeeper cannot find junit

Bug

Closed

Major

Fixed

Unassigned

Flavio Paiva Junqueira

24/Nov/09 04:48

26/Mar/10 13:31

02/Feb/10 09:47

contrib-bookkeeper

0

1

When trying to compile bookkeeper, the compiler complains that it can't find junit. I suspect that this is related to the fact that zookeeper now fetches junit using ivy.

47710

No Perforce job exists for this issue.

0

32981

10 years, 8 weeks, 2 days ago

0|i060b3:

ZooKeeper

ZOOKEEPER-591

The C Client cannot exit properly in some situation

Bug

Closed

Blocker

Fixed

Mahadev Konar

Qian Ye

23/Nov/09 04:35

26/Mar/10 13:25

17/Mar/10 21:45

3.2.1

3.3.0

c client

0

2

Linux db-passport-test05.vm 2.6.9_5-4-0-5 #1 SMP Tue Apr 14 15:56:24 CST 2009 x86_64 x86_64 x86_64 GNU/Linux

The following code produce a situation, where the C Client can not exit properly,

#include "include/zookeeper.h"

void default_zoo_watcher(zhandle_t *zzh, int type, int state, const char *path, void* context){
int zrc = 0;
struct String_vector str_vec = {0, NULL};
printf("in the default_zoo_watcher\n");
zrc = zoo_wget_children(zzh, "/mytest", default_zoo_watcher, NULL, &str_vec);
printf("zoo_wget_children, error: %d\n", zrc);

return;
}

int main()
{
int zrc = 0;
int buff_len = 10;
char buff[10] = "hello";
char path[512];
struct Stat stat;
struct String_vector str_vec = {0, NULL};

zhandle_t *zh = zookeeper_init("10.81.20.62:2181", NULL, 30000, 0, 0, 0);
zrc = zoo_create(zh, "/mytest", buff, 10, &ZOO_OPEN_ACL_UNSAFE, 0, path, 512);
printf("zoo_create, error: %d\n", zrc);

zrc = zoo_wget_children(zh, "/mytest", default_zoo_watcher, NULL, &str_vec);
printf("zoo_wget_children, error: %d\n", zrc);

zrc = zoo_create(zh, "/mytest/test1", buff, 10, &ZOO_OPEN_ACL_UNSAFE, 0, path, 512);
printf("zoo_create, error: %d\n", zrc);

zrc = zoo_wget_children(zh, "/mytest", default_zoo_watcher, NULL, &str_vec);
printf("zoo_wget_children, error: %d\n", zrc);

zrc = zoo_delete(zh, "/mytest/test1", -1);

printf("zoo_delete, error: %d\n", zrc);
zookeeper_close(zh);
return 0;
}

running this code can cause the program hang at zookeeper_close(zh);(line 38). using gdb to attach the process, I found that the main thread is waiting for do_completion thread to finish,
(gdb) bt
#0 0x000000302b806ffb in pthread_join () from /lib64/tls/libpthread.so.0
#1 0x000000000040de3b in adaptor_finish (zh=0x515b60) at src/mt_adaptor.c:219
#2 0x00000000004060ba in zookeeper_close (zh=0x515b60) at src/zookeeper.c:2100
#3 0x000000000040220b in main ()

and the thread which handle the zoo_wget_children(in the default_zoo_watcher) is waiting for sc->cond.
(gdb) thread 2
[Switching to thread 2 (Thread 1094719840 (LWP 25093))]#0 0x000000302b8089aa in pthread_cond_wait@@GLIBC_2.3.2 ()
from /lib64/tls/libpthread.so.0
(gdb) bt
#0 0x000000302b8089aa in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0
#1 0x000000000040d88b in wait_sync_completion (sc=0x5167f0) at src/mt_adaptor.c:82
#2 0x00000000004082c9 in zoo_wget_children (zh=0x515b60, path=0x40ebc0 "/mytest", watcher=0x401fd8 <default_zoo_watcher>, watcherCtx=Variable "watcherCtx" is not available.)
at src/zookeeper.c:2884
#3 0x0000000000402037 in default_zoo_watcher ()
#4 0x000000000040d664 in deliverWatchers (zh=0x515b60, type=4, state=3, path=0x515100 "/mytest", list=0x5177d8) at src/zk_hashtable.c:274
#5 0x0000000000403861 in process_completions (zh=0x515b60) at src/zookeeper.c:1631
#6 0x000000000040e1b5 in do_completion (v=Variable "v" is not available.) at src/mt_adaptor.c:333
#7 0x000000302b80610a in start_thread () from /lib64/tls/libpthread.so.0
#8 0x000000302afc6003 in clone () from /lib64/tls/libc.so.6
#9 0x0000000000000000 in ?? ()

here, a deadlock presents.

47711

No Perforce job exists for this issue.

9

32982

19/Jan/10 12:50 PM

10 years, 2 weeks ago

Reviewed

Zookeeper C Client

0|i060bb:

ZooKeeper

ZOOKEEPER-590

review logging to ensure that session related messages include session id

Bug

Open

Major

Unresolved

Patrick D. Hunt

23/Nov/09 02:01

14/Dec/19 06:07

3.7.0

server

0

1

when the server is logging session related log messages it must include the session id in hex form

this greatly simplifies debugging - being able to relate a session message back to a particular session. otw there's
too much going on and there is no way to determine what messages are related to a particular session

newbie

70779

No Perforce job exists for this issue.

0

32983

10 years, 2 weeks, 6 days ago

0|i060bj:

ZooKeeper

ZOOKEEPER-589

When create a znode, a NULL ACL parameter cannot be accepted

Bug

Closed

Major

Fixed

Benjamin Reed

Qian Ye

22/Nov/09 21:47

26/Mar/10 13:25

08/Feb/10 17:38

3.2.1

3.3.0

c client

0

2

Linux db-passport-test05.vm 2.6.9_5-4-0-5 #1 SMP Tue Apr 14 15:56:24 CST 2009 x86_64 x86_64 x86_64 GNU/Linux

In the comments of client C API which associated with creating znode, eg. zoo_acreate, it is said that the initial ACL of the node "if null, the ACL of the parent will be used". However, the it doesn't work. When execute this kind of request at the server side, it raises InvalidACLException. The source code show that, the function fixupACL return false when it get a null ACL.

47712

No Perforce job exists for this issue.

1

32984

01/Feb/10 2:39 PM

10 years, 7 weeks, 2 days ago

Reviewed

Zookeeper C Client

0|i060br:

ZooKeeper

ZOOKEEPER-588

remove unnecessary/annoying log of tostring error in Request.toString()

Bug

Closed

Minor

Fixed

Patrick D. Hunt

21/Nov/09 18:23

07/May/10 13:39

11/Dec/09 15:24

3.2.1

3.3.0

server

0

2

Why are we logging this? It's unnecessary and just annoying afaict. We should remove it entirely.

2009-11-18 05:37:29,312 WARN org.apache.zookeeper.server.Request: Ignoring exception during toString
java.nio.BufferUnderflowException
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
at java.nio.ByteBuffer.get(ByteBuffer.java:675)
at org.apache.zookeeper.server.Request.toString(Request.java:199)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at org.apache.zookeeper.server.quorum.CommitProcessor.processRequest(CommitProcessor.java:167)
at org.apache.zookeeper.server.quorum.FollowerRequestProcessor.run(FollowerRequestProcessor.java:68)

logging

47713

No Perforce job exists for this issue.

1

32985

24/Nov/09 12:38 PM

10 years, 15 weeks, 3 days ago

Reviewed

0|i060bz:

ZooKeeper

ZOOKEEPER-587

client should log timeout negotiated with server

Bug

Closed

Major

Fixed

Patrick D. Hunt

21/Nov/09 18:21

07/May/10 13:40

11/Dec/09 16:46

3.2.1

3.3.0

c client, java client

0

2

ZOOKEEPER-593

The ZK client should log the timeout negotiated with the server if the time is different than the timeout parameter specified by the client.

logging

47714

No Perforce job exists for this issue.

2

32986

21/Nov/09 7:08 PM

10 years, 15 weeks, 3 days ago

Reviewed

0|i060c7:

ZooKeeper

ZOOKEEPER-586

c client does not compile under cygwin

Bug

Closed

Major

Fixed

Patrick D. Hunt

20/Nov/09 20:18

26/Mar/10 13:25

16/Mar/10 19:44

3.2.1

3.3.0

c client

0

2

ZOOKEEPER-674

the c client fails to compile under cygwin

47715

No Perforce job exists for this issue.

6

32987

19/Feb/10 8:49 PM

10 years, 2 weeks, 2 days ago

Reviewed

0|i060cf:

ZooKeeper

ZOOKEEPER-585

Update README for zkpython in 3.2.2

Improvement

Closed

Minor

Fixed

Henry Robinson

19/Nov/09 19:02

26/Mar/10 13:29

20/Nov/09 14:23

3.2.2

contrib-bindings

0

1

zkpython has a few improvements going into 3.2.2, and its README needs a short update to reflect this.

47716

No Perforce job exists for this issue.

1

33432

20/Nov/09 2:23 PM

10 years, 18 weeks, 6 days ago

Reviewed

0|i0633b:

ZooKeeper

ZOOKEEPER-584

ZooKeeper service instance should be registered in the OSGi registry

New Feature

Open

Major

Unresolved

Unassigned

Alan Cabrera

19/Nov/09 10:16

01/May/13 22:29

server

2

4

ZOOKEEPER-425

When Zookeeper is booted in an OSGi framework by {{ManagedService}} it would be quite handy to have {{ManagedService}} register a management interface for that ZooKeeper instance into the OSGi service registry.

214139

No Perforce job exists for this issue.

0

42186

26/Nov/09 9:13 AM

10 years, 17 weeks, 3 days ago

0|i07l3r:

ZooKeeper

ZOOKEEPER-583

on resync client should generate session expired exception if there is no server in cluster with acceptable zxid

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

17/Nov/09 17:47

14/Dec/19 06:08

3.1.1, 3.2.1

3.7.0

c client, java client

0

2

Both the c and java clients attempt to connect to a server in the cluster by iterating through
a randomized list of servers as listed in the connect string passed to the zookeeper_init (c)
or ZooKeeper constructor (java). The clients do this indefinitely, until successfully connecting
to a server or until the client is close()ed. Additionally if a client is disconnected from a server
it will attempt to reconnect to another server in the cluster, in this case it will only connect
to a server that has the same, or higher, zxid as seen by the client on the previous server that
it was connected to (this ensures that the client never sees old data).

In some weird cases (in particular where operators reset the server database, clearing out the
existing snapshots and txnlogs) existing clients will now see a much lower zxid (due to the
epoch number being reset) regardless of the server that the client attempts to connect to. In this
case the current client will iterate essentially forever.

Instead the client should throw session expired in this case (notify any watchers). After iterating
through all of the servers in the list, if none of the servers have an acceptable zxid the client
should expire the session and shut down the handle. This will ensure that the client will eventually
shutdown in this unusual, but possible (esp with server operators who don't also control the
clients) situation.

70724

No Perforce job exists for this issue.

0

32988

10 years, 3 weeks, 6 days ago

0|i060cn:

ZooKeeper

ZOOKEEPER-582

ZooKeeper can revert to old data when a snapshot is created outside of normal processing

Bug

Closed

Blocker

Fixed

Mahadev Konar

Benjamin Reed

17/Nov/09 15:56

26/Mar/10 13:25

20/Nov/09 17:30

3.1.1, 3.2.1

3.1.2, 3.2.2, 3.3.0

server

0

3

when zookeeper starts up it will restore the most recent state (latest zxid) it finds in the data directory. unfortunately, in the quorum version of zookeeper updates are logged using an epoch based on the latest log file in a directory. if there is a snapshot with a higher epoch than the log files, the zookeeper server will start logging using an epoch one higher than the highest log file.

so if a data directory has a snapshot with an epoch of 27 and there are no log files, zookeeper will start logging changes using epoch 1. if the cluster restarts the state will be restored from the snapshot with the epoch of 27, which in effect, restores old data.

normal operation of zookeeper will never result in this situation.

this does not effect standalone zookeeper.

a fix should make sure to use an epoch one higher than the current state, whether it comes from the snapshot or log, and should include a sanity check to make sure that a follower never connects to a leader that has a lower epoch than its own.

47717

No Perforce job exists for this issue.

9

32989

17/Nov/09 4:37 PM

10 years, 18 weeks, 5 days ago

fixed bug in zookeeper that can lead zookeeper to revert to old data when a snapshot is created without a corresponding log.

Reviewed

0|i060cv:

ZooKeeper

ZOOKEEPER-581

peerType in configuration file is redundant

Improvement

Open

Minor

Unresolved

Unassigned

Benjamin Reed

17/Nov/09 11:23

17/Nov/09 11:24

1

3

ZOOKEEPER-368

to configure a machine to be an observer you must add a peerType=observer to the configuration file and an observer tag to the server list. this is redundant. if the observer tag is on the entry of a machine it should know it is an observer without needing the peerType tag.

on the other hand, do we really need the observers in the server list? they don't vote.

214138

No Perforce job exists for this issue.

0

42187

10 years, 19 weeks, 2 days ago

0|i07l3z:

ZooKeeper

ZOOKEEPER-580

Document reasonable limits on the size and shape of data for a zookeeper ensemble.

Improvement

Open

Major

Unresolved

Unassigned

bryan thompson

17/Nov/09 10:54

21/Aug/13 12:55

documentation

1

5

I would like to have documentation which clarifies the reasonable limits on the size and shape of data in a zookeeper ensemble. Since all zookeeper nodes and their data are replicated on each peer in an ensemble, there will be a machine limit on the amount of data in a zookeeper instance, but I have not seen any guidance on the estimation of that machine limit. Presumably the machine limits are primarily determined by the amount of heap available to the JVM before swapping sets in, however there might well be other limits which are less obvious in terms of the #of children per node and the depth of the node hierarchy (in addition to the already documented limit on the amount of data in a node). There may also be interactions with the hierarchy depth and performance, which I have not seen detailed anywhere.

Guidance regarding pragmatic and machine limits would be helpful is choosing designs using zookeeper which can scale. For example, if metadata about each shard of a partitioned database architecture is mapped onto a distinct znode in zookeeper, then there could be an very large number of znodes for a large database deployment. While this would make it easy to reassign shards to services dynamically, the design might impose an unforeseen limit on the #of shards in the database. A similar concern would apply to an attempt to maintain metadata about each file in a distributed file system.

Issue [ZOOKEEPER-272] described some problems when nodes have a large number #of children. However, it did not elaborate on whether the change to an Iterator model would break the atomic semantics of the List<String> of children or if the Iterator would be backed by a snapshot of the children as it existed at the time the iterator was requested, which would put a memory burden on the ensemble. This raises the related question of when designs which work around scaling limits in zookeeper might break desirable semantics, primarily the ability to have a consistent view of the distributed state.

Put another way, are there anti-patterns for zookeeper relating to scalability? Too many children? Too much depth? Avoid decomposing large numbers of children into hierarchies? Etc.

214137

No Perforce job exists for this issue.

0

42188

25/Jan/12 7:57 PM

6 years, 31 weeks, 1 day ago

0|i07l47:

ZooKeeper

ZOOKEEPER-579

zkpython needs more test coverage for ACL code paths

Improvement

Closed

Major

Fixed

Henry Robinson

12/Nov/09 18:26

26/Mar/10 13:25

04/Mar/10 22:48

3.2.1

3.3.0

contrib-bindings

0

2

zkpython's tests don't do a good enough job of exercising the ACL code paths. A few new tests that confirm that setACL and friends are working correctly are needed.

47718

No Perforce job exists for this issue.

2

33433

08/Feb/10 7:14 PM

10 years, 3 weeks, 6 days ago

Reviewed

0|i0633j:

ZooKeeper

ZOOKEEPER-578

ResponderThread should be able to use TCP or UDP

Improvement

Resolved

Major

Won't Fix

Henry Robinson

11/Nov/09 13:37

03/Apr/12 17:16

3.2.1

server

0

1

The ResponderThread, which responds to inquiries for the current leader, currently only uses UDP. It also only runs for electionAlg=0.

Observers will eventually require that a ResponderThread runs for all election types. However, this introduces a UDP dependency which some installations do not want. This would also allow such installations to use electionAlg=0 (although this is not a big win as it is the least sophisticated election algorithm).

Therefore we should be able to toggle ResponderThread to use either TCP or UDP. Since UDP is more performant, it probably makes sense to retain it. So I propose to choose between the two at startup time using a configuration flag responderTCP=true.

Fixing this issue exposed ZOOKEEPER-569, on which this JIRA depends.

214136

No Perforce job exists for this issue.

0

33434

7 years, 51 weeks, 2 days ago

0|i0633r:

ZooKeeper

ZOOKEEPER-577

LeaderElection code hardcodes majority quorums in at least two places

Bug

Open

Major

Unresolved

Henry Robinson

11/Nov/09 13:30

01/May/13 22:29

3.2.1

server

0

ZOOKEEPER-372

See e.g. lookForLeader in LeaderElection.java and termPredicate in AuthFastLeaderElection.java

Should use containsQuorum.

214135

No Perforce job exists for this issue.

0

32990

12/Nov/09 10:34 AM

10 years, 20 weeks ago

0|i060d3:

ZooKeeper

ZOOKEEPER-576

docs need to be updated for session moved exception and how to handle it

Bug

Closed

Major

Fixed

Benjamin Reed

Mahadev Konar

11/Nov/09 13:08

26/Mar/10 13:25

20/Nov/09 14:47

3.2.2, 3.3.0

0

2

the handling and implications of session moved exception should be documented.

47719

No Perforce job exists for this issue.

2

32991

19/Nov/09 7:04 PM

10 years, 18 weeks, 5 days ago

Reviewed

0|i060db:

ZooKeeper

ZOOKEEPER-575

remove System.exit calls to make the server more container friendly

Improvement

Open

Major

Unresolved

Andrew Finnell

Patrick D. Hunt

11/Nov/09 00:54

14/Dec/19 06:09

3.4.0

3.7.0

server

2

6

ZOOKEEPER-1072

There are a handful of places left in the code that still use System.exit, we should remove these to make the server
more container friendly.

There are some legitimate places for the exits - in *Main.java for example should be fine - these are the command
line main routines. Containers should be embedding code that runs just below this layer (or we should refactor
so that it would).

The tricky bit is ensuring the server shuts down in case of an unrecoverable error occurring, afaik these are the
locations where we still have sys exit calls.

45

No Perforce job exists for this issue.

4

42189

03/Sep/11 11:37 AM

6 years, 6 days ago

0|i07l4f:

ZooKeeper

ZOOKEEPER-574

the documentation on snapcount in the admin guide has the wrong default

Bug

Closed

Minor

Fixed

Patrick D. Hunt

10/Nov/09 19:43

26/Mar/10 13:25

22/Jan/10 19:31

3.3.0

documentation

0

2

I believe it's 100k, not 10k

-----------------------
snapCount
(Java system property: zookeeper.snapCount)

Clients can submit requests faster than ZooKeeper can process them, especially if there are a lot of clients. To prevent ZooKeeper from running out of memory due to queued requests, ZooKeeper will throttle clients so that there is no more than globalOutstandingLimit outstanding requests in the system. The default limit is 1,000.ZooKeeper logs transactions to a transaction log. After snapCount transactions are written to a log file a snapshot is started and a new transaction log file is started. The default snapCount is 10,000.

47720

No Perforce job exists for this issue.

1

32992

22/Jan/10 7:24 PM

10 years, 9 weeks, 6 days ago

Reviewed

0|i060dj:

ZooKeeper

ZOOKEEPER-573

the dump 4letterword is not formatting sessionids in hex

Bug

Closed

Major

Fixed

Patrick D. Hunt

09/Nov/09 20:05

26/Mar/10 13:25

20/Jan/10 15:17

3.2.1

3.3.0

server

0

2

ZOOKEEPER-566, ZOOKEEPER-572

47721

No Perforce job exists for this issue.

0

32993

10 years, 10 weeks, 1 day ago

0|i060dr:

ZooKeeper

ZOOKEEPER-572

add ability for operator to examine state of watches currently registered with a server

Improvement

Closed

Major

Fixed

Patrick D. Hunt

09/Nov/09 19:39

01/Jul/15 15:24

25/Jan/10 19:18

3.3.0

jmx, server

0

1

ZOOKEEPER-573, ZOOKEEPER-2227

it may be useful for an operator to examine the watches registered with a server by the various connected sessions

seems useful to allow:

1) watches on a session
2) watches on a path
3) all watches?

command port and JMX.

47722

No Perforce job exists for this issue.

4

33435

20/Jan/10 2:20 PM

10 years, 9 weeks, 1 day ago

Add four letter words for monitoring/administering watches on zookeeper nodes. See zookeeperAdmin.pdf/html for more info.

Reviewed

0|i0633z:

ZooKeeper

ZOOKEEPER-571

support balancing of client load across servers in an ensemble

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

09/Nov/09 19:34

10/Apr/17 12:17

quorum, server

0

4

Currently the ensemble does not ensure a balanced load across servers in an ensemble. Clients randomly connect to
a server, which typically balances the number of sessions. However there are problems with this:

1) session count is balanced, but not session load
2) if server A goes down all of the sessions on that server migrate to other servers in the cluster randomly, this is fine, however
when server A comes back into service it will have no sessions, and migration of sessions from other servers may take time

The quorum should probably have some way of broadcasting load, and occasionally re-balance the sessions based on
this information. Might be tricky though, want to ensure that we aren't constantly ping-ponging sessions to servers.
Probably need some hysteresis as well as limit the frequency. Real time tuning would need to be supported.

214134

No Perforce job exists for this issue.

0

42190

10/Apr/17 12:17 PM

2 years, 49 weeks, 3 days ago

0|i07l4n:

ZooKeeper

ZOOKEEPER-570

AsyncHammerTest is broken, callbacks need to validate rc parameter

Bug

Closed

Critical

Fixed

Patrick D. Hunt

08/Nov/09 01:37

26/Mar/10 13:25

08/Nov/09 17:46

3.2.1

3.3.0

tests

0

2

the asynchammertest is not validating the rc in the callback, more serious is that it is using path in the create callback
to delete the node, rather than name (which is important in the case of a sequential node creation as in this case)

47723

No Perforce job exists for this issue.

2

32994

08/Nov/09 2:44 AM

10 years, 18 weeks, 6 days ago

Reviewed

0|i060dz:

ZooKeeper

ZOOKEEPER-569

Failure of elected leader can lead to never-ending leader election

Bug

Closed

Major

Fixed

Henry Robinson

06/Nov/09 14:47

26/Mar/10 13:25

20/Feb/10 09:27

3.3.0

0

3

ZOOKEEPER-653

It is possible for basic LeaderElection to enter a situation where it never terminates.

As an example, consider a three node cluster A, B and C.

1. In the first round, A votes for A, B votes for B and C votes for C
2. Since C > B > A, all nodes resolve to vote for C in the second round as there is no first round winner
3. A, B vote for C, but C fails.
4. C is not elected because neither A nor B hear from it, and so votes for it are discarded
5. A and B never reset their votes, despite not hearing from C, so continue to vote for it ad infinitum.

Step 5 is the bug. If A and B reset their votes to themselves in the case where the heard-from vote set is empty, leader election will continue.

I do not know if this affects running ZK clusters, as it is possible that the out-of-band failure detection protocols may cause leader election to be restarted anyhow, but I've certainly seen this in tests.

I have a trivial patch which fixes it, but it needs a test (and tests for race conditions are hard to write!)

47724

No Perforce job exists for this issue.

6

32995

13/Jan/10 4:36 AM

10 years, 5 weeks, 4 days ago

0|i060e7:

ZooKeeper

ZOOKEEPER-568

SyncRequestProcessor snapping too frequently - counts non-log events as log events

Bug

Closed

Major

Fixed

Patrick D. Hunt

02/Nov/09 20:38

15/Mar/19 17:40

07/Nov/09 03:08

3.2.1

3.3.0

0

2

ZOOKEEPER-3316

Noticed the following issues in SyncRequestProcessor

1) logCount is incremented even for non-log events (say getData)
txnlog should return indication if request was logged or not (if hdr ==null it returns)

also:

2) move r.nextInt below logCount++ (ie if an actual log event)
3) fix indentation after txnlog.append (for some reason has unnecessary 4 char indent)

47725

No Perforce job exists for this issue.

1

32996

03/Nov/09 6:33 PM

10 years, 20 weeks, 4 days ago

Reviewed

0|i060ef:

ZooKeeper

ZOOKEEPER-567

javadoc for getchildren2 needs to mention "new in 3.3.0"

Bug

Closed

Major

Fixed

Patrick D. Hunt

01/Nov/09 00:41

26/Mar/10 13:25

12/Nov/09 15:03

3.3.0

c client, java client

0

2

ZOOKEEPER-555

the javadoc/cdoc for getchildren2 needs to mention that the methods are "new in 3.3.0"

47726

No Perforce job exists for this issue.

1

32997

06/Nov/09 5:34 PM

10 years, 19 weeks, 5 days ago

Reviewed

0|i060en:

ZooKeeper

ZOOKEEPER-566

"reqs" four letter word (command port) returns no information

Bug

Closed

Critical

Fixed

Patrick D. Hunt

29/Oct/09 01:27

26/Mar/10 13:25

12/Nov/09 14:54

3.2.1

3.3.0

0

2

ZOOKEEPER-573

the four letter word "reqs" doesn't do anything - it always returns empty data. Seems that "outstanding" field is always empty and never set.

we should remove outstanding and also update the reqs code to correctly output the outstanding requests (if not possible then remove the cmd and update docs - although this is very useful command, hate to see us lose it)

47727

No Perforce job exists for this issue.

1

32998

10/Nov/09 2:34 AM

10 years, 19 weeks, 5 days ago

Committed revision 835515.

Reviewed

0|i060ev:

ZooKeeper

ZOOKEEPER-565

Revisit some java client log messages

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Jean-Daniel Cryans

28/Oct/09 16:47

07/May/10 13:40

06/Nov/09 23:52

3.2.1

3.3.0

0

2

ZOOKEEPER-558

As discussed during the 10/23 meeting, some messages in the java client logs are mixing up terms from different levels. For example:

{code}
2009-10-14 15:12:43,566 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x1244f619478000d to sun.nio.ch.SelectionKeyImpl@15e32c4
java.io.IOException: Session Expired
at org.apache.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:589)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:709)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
{code}

Which session are we talking about in the first line? Now I know that it's a network-related session and not the ZK one, but I've seen many of our users getting confused over those lines.

logging

47728

No Perforce job exists for this issue.

0

33436

29/Oct/09 6:31 PM

10 years, 20 weeks, 5 days ago

Reviewed

0|i06347:

ZooKeeper

ZOOKEEPER-564

Give more feedback on that current flow of events in java client logs

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Jean-Daniel Cryans

28/Oct/09 16:42

07/May/10 13:40

06/Nov/09 23:52

3.2.1

3.3.0

0

1

ZOOKEEPER-558

As discussed during the 10/23 meeting, one issue we have in debugging ZK client logs with HBase is that we have a hard time following the flow of events. It may be obvious for a ZK dev, but in our POV that kind of trace isn't very intuitive:

{code}
2009-09-27 15:41:10,776 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server ...
2009-09-27 15:41:10,776 INFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/ ... remote=...
2009-09-27 15:41:10,776 INFO org.apache.zookeeper.ClientCnxn: Server connection successful
2009-09-27 15:41:10,784 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x0 to sun.nio.ch.SelectionKeyImpl@2c9b42e6
{code}

This excerpt is just an example. We would like to see something like a numbering of the events and possibly, in the case of an exception, at which point did it went wrong and what's the next step.

logging

47729

No Perforce job exists for this issue.

0

33437

30/Oct/09 2:14 AM

10 years, 20 weeks, 5 days ago

Reviewed

0|i0634f:

ZooKeeper

ZOOKEEPER-563

ant test for recipes is broken.

Bug

Closed

Major

Fixed

Mahadev Konar

28/Oct/09 13:09

26/Mar/10 13:25

29/Oct/09 00:02

3.3.0

build

0

2

With ZOOKEEPER-529 checked in, ant test for recipes broke. Its a minor change to the build for including librariries from the new location where jars are downloaded by ivy.

47730

No Perforce job exists for this issue.

1

32999

28/Oct/09 6:56 PM

10 years, 22 weeks ago

Reviewed

0|i060f3:

ZooKeeper

ZOOKEEPER-562

c client can flood server with pings if tcp send queue filled

Bug

Closed

Blocker

Fixed

Benjamin Reed

Patrick D. Hunt

26/Oct/09 16:45

26/Mar/10 13:25

29/Oct/09 17:35

3.2.1

3.1.2, 3.2.2, 3.3.0

c client

0

2

The c client can flood the server with pings if the tcp queue is filled.

Say the cluster is overloaded and shuts down the recv processing

a c client can send a ping, but since last_send is only updated on successful pushing of data into the
socket, if flush_send_queue fails to send any data (send_buffer returns 0) then last_send is not updated
and zookeeper_interest will again send a ping the next time it is woken - which could be 0 if recv_to is close
to 0, easily could happen if server is not sending data to the client.

47731

No Perforce job exists for this issue.

1

33000

27/Oct/09 1:34 AM

10 years, 21 weeks, 6 days ago

Basically the problem here is that the client gets confused, it tries to send a ping to the server but since the tcp queue is full it's unable to do so. The logic responsible for sending occasional pings based on the timeout gets confused by this, and ends up flooding the server with pings. Eventually this clears up, however it can result in increased load on the server and instability for the effected client.

Reviewed

0|i060fb:

ZooKeeper

ZOOKEEPER-561

inefficient buffer management, inefficient gc, in NIOServer request handling

Improvement

Closed

Major

Invalid

Unassigned

Patrick D. Hunt

26/Oct/09 13:24

26/Mar/10 13:25

06/Nov/09 17:14

3.3.0

server

0

2

for every request we allocate a buffer to read the body. in most cases the request (say for get) will be very small, but we
always allocate a new buffer to store the request.

There should be a way to improve the buffer handling here. In particular it seems that if we have a buffer of reasonable
length (1k) preallocated we could use this to service all requests <= 1k and only allocate when requests are larger. seems
like this would handle a large fraction of the requests we typically see (except for large sets for example)

214133

No Perforce job exists for this issue.

0

33438

10 years, 20 weeks, 6 days ago

0|i0634n:

ZooKeeper

ZOOKEEPER-560

inefficient request handling in NIOServerCnxn

Improvement

Closed

Major

Fixed

Patrick D. Hunt

26/Oct/09 13:21

26/Mar/10 13:25

06/Nov/09 23:53

3.3.0

server

0

1

ZOOKEEPER-558

the server doIO code, specifically for read is inefficiently handling requests.

dio does the following for reads

if new request received: read length and return
else read request body and return
....

as a result we poll the selector twice for each request from a client.

we should just read the request fully, reducing selector polling by half

47732

No Perforce job exists for this issue.

0

33439

10 years, 20 weeks, 5 days ago

Reviewed

0|i0634v:

ZooKeeper

ZOOKEEPER-559

valgrind warnings running zkpython bindings

Bug

Resolved

Major

Duplicate

Lei Zhang

Patrick D. Hunt

26/Oct/09 13:11

10/Oct/13 13:32

3.3.0

contrib-bindings

0

1

I'm seeing some weird behavior running zk-latencies.py
http://github.com/phunt/zk-smoketest

don't know if it's related to zkbindings itself, but I ran valgrind to see if it noticed any issues. see attached.

afaict these issues are related to zkpython binding, however I'm not sure. I did run valgrind against the
zookeeper c library tests and these issues were not highlighted. So I'm thinking this is zkpython errors, however
I'm not 100% sure.

Henry can you take a look?

70763

No Perforce job exists for this issue.

1

33001

10/Mar/10 2:51 AM

6 years, 24 weeks ago

0|i060fj:

ZooKeeper

ZOOKEEPER-558

server "sent" stats not being updated

Bug

Closed

Critical

Fixed

Patrick D. Hunt

24/Oct/09 20:08

26/Mar/10 13:25

06/Nov/09 19:48

3.2.1

3.3.0

server

0

2

ZOOKEEPER-560, ZOOKEEPER-564, ZOOKEEPER-565

the server and connection "sent" stat is not being updated. if you run "stat" on the client port the sent packets is much lower than it should be

seems that sendbuffer is not updating the stats when it shortcircuits the send.

47733

No Perforce job exists for this issue.

2

33002

02/Nov/09 8:11 PM

10 years, 20 weeks, 4 days ago

Reviewed

0|i060fr:

ZooKeeper

ZOOKEEPER-557

Upload Zookeeper jars to a public maven repository

Bug

Closed

Major

Duplicate

Unassigned

Cyril Lakech

24/Oct/09 17:03

26/Mar/10 13:25

06/Nov/09 16:24

3.3.0

1

3

There are many cool release of hadoop zookeeper and this project is an apache project, as the maven project.
But the released jars must be download manually and then deploy to a private repository before they can be used by developer using maven2.

Please could you upload the zookeeper jars on the public maven2 repository ?

Of course, we can help to deploy those artifact if necessary.

214132

No Perforce job exists for this issue.

0

33003

06/Nov/09 4:24 PM

10 years, 20 weeks, 6 days ago

0|i060fz:

ZooKeeper

ZOOKEEPER-556

Startup messages should account for common error of missing leading slash in config files

Improvement

Resolved

Major

Fixed

Thomas Koch

Ted Dunning

22/Oct/09 18:16

17/Sep/11 06:56

16/Sep/11 20:43

3.5.0

server

0

2

It would be nice if the startup noticed directories without a leading slash in the config file. That is worth a warning.

Moreover, if that directory exists looking from root, but not looking from the current directory, a very serious warning is in order.

34431

No Perforce job exists for this issue.

4

33440

06/Sep/11 7:20 AM

8 years, 27 weeks, 5 days ago

Reviewed

0|i06353:

ZooKeeper

ZOOKEEPER-555

Add stat information to GetChildrenResponse

Improvement

Closed

Minor

Fixed

Árni Már Jónsson

15/Oct/09 06:58

26/Mar/10 13:25

30/Oct/09 19:09

3.3.0

c client, contrib-bindings, java client, server

0

1

ZOOKEEPER-567

GetChildren() is the only non-create/delete API which does not include the node stat information. I propose that the definition of GetChildren() should be:

class GetChildrenResponse {
vector<ustring> children;
org.apache.zookeeper.data.Stat stat;
}

There is a trivial fix to the server (FinalRequestProcessor.java): rsp = new GetChildrenResponse(children, stat);

And something similar to the client library.

47734

No Perforce job exists for this issue.

4

33441

15/Oct/09 11:12 AM

10 years, 21 weeks, 5 days ago

This change is backward incompatible with server versions prior to 3.3.0 given that the wire protocol has changed. However at the client API level this is a b/w compatible change (added new api methods).

Changes required to have Stat returned with get_children().

Incompatible change, Reviewed

0|i0635b:

ZooKeeper

ZOOKEEPER-554

zkpython can segfault when statting a deleted node

Bug

Closed

Major

Fixed

Henry Robinson

14/Oct/09 18:55

26/Mar/10 13:25

19/Oct/09 15:43

3.2.2, 3.3.0

contrib-bindings

0

2

C client returns NULL for stat object for deleted nodes. zookeeper.c blindly dereferences it. Segfault.

47735

No Perforce job exists for this issue.

1

33004

14/Oct/09 9:56 PM

10 years, 23 weeks, 3 days ago

Reviewed

0|i060g7:

ZooKeeper

ZOOKEEPER-553

C client's zoo_state function returns unknown state 0

Bug

Open

Minor

Unresolved

Unassigned

Steven K. Wong

13/Oct/09 20:47

14/Dec/19 06:07

3.2.1

3.7.0

c client, contrib-bindings

0

Using zkpython with ZK 3.2.1 release:

import zookeeper as z
zh = z.init(...)
z.state(zh) # returns 3 == z.CONNECTED_STATE
# kill standalone ZK server
z.state(zh) # returns 0 == ???

The problem is that 0 is not a state defined by the C client's zookeeper.[ch]. Perhaps 0 should've been defined as something like DISCONNECTED_STATE? (Java's KeeperState.Disconnected is 0, if that matters.)

If the fix is to define 0 as a new state, changes will be needed to both the C client and zkpython. Not sure about other bindings.

68119

No Perforce job exists for this issue.

0

33005

05/Mar/10 6:23 PM

8 years, 37 weeks, 1 day ago

0|i060gf:

ZooKeeper

ZOOKEEPER-552

add performance benchmark/docs for synchronous operations

Task

Open

Major

Unresolved

Benjamin Reed

Patrick D. Hunt

13/Oct/09 17:50

14/Dec/19 06:06

3.7.0

c client, java client, server

0

we currently benchmark async operations, but not sync. it would be good to benchmark sync operations so that users
know what to expect/tradeoffs.

Also, afaik we currently only benchmark java client - we should benchmark the c client as well (if only to ensure perf is comperable)

70760

No Perforce job exists for this issue.

0

42191

16/Oct/09 1:32 PM

10 years, 3 weeks, 6 days ago

0|i07l4v:

ZooKeeper

ZOOKEEPER-551

unnecessary SetWatches message on new session

Bug

Closed

Minor

Fixed

Patrick D. Hunt

13/Oct/09 00:10

26/Mar/10 13:25

08/Nov/09 16:42

3.2.1

3.3.0

c client, java client

0

2

The java client is sending a SetWatches message even on a new session (always empty). Additionally SetWatches is called
even in the case of re-establishing session, however no watches are set. The code should check for watches before
sending this (ie don't send empty setwatches). I see this on java, investigate c as well.

47736

No Perforce job exists for this issue.

1

33006

06/Nov/09 6:19 PM

10 years, 20 weeks, 3 days ago

0|i060gn:

ZooKeeper

ZOOKEEPER-550

Java Queue Recipe

New Feature

Closed

Minor

Fixed

Steven Cheng

09/Oct/09 18:59

26/Mar/10 13:25

12/Nov/09 16:27

3.2.1

3.3.0

java client

0

2

This patch adds a recipe for creating a distributed queue with ZooKeeper similar to the WriteLock recipe and some sequential tests. This early attempt follows the Java BlockingQueue interface, though it doesn't implement it since I don't think there's a good reason for it to be Iterable.

47737

No Perforce job exists for this issue.

8

33442

11/Oct/09 2:29 PM

10 years, 19 weeks, 5 days ago

Reviewed

0|i0635j:

ZooKeeper

ZOOKEEPER-549

Refactor Followers and related classes into a Peer->Follower hierarchy in preparation for Observers

Improvement

Closed

Major

Fixed

Henry Robinson

08/Oct/09 19:19

26/Mar/10 13:25

30/Oct/09 12:20

3.2.1

3.3.0

quorum, server

0

3

For the Observers patch (ZOOKEEPER-368), a lot of functionality is shared between Followers and Observers. To avoid copying code, it makes sense to push the common code into a parent Peer class and specialise it for Followers and Observers. At the same time, some of the lengthier methods in Follower can be broken up to make the code more readable.

47738

No Perforce job exists for this issue.

4

33443

12/Oct/09 2:55 AM

10 years, 21 weeks, 5 days ago

Refactor followers code into Learner from which followers and observers will inherit.

Reviewed

0|i0635r:

ZooKeeper

ZOOKEEPER-548

zookeeper.ZooKeeperException not added to the module in zkpython

Bug

Closed

Critical

Invalid

Patrick D. Hunt

08/Oct/09 18:59

26/Mar/10 13:25

09/Oct/09 00:35

3.3.0

contrib-bindings

0

1

ZooKeeperException is not being added to the zookeeper module in zookeeper.c (zkpython). The other exceptions
are added but not ZooKeeperException. Sorry, I missed this in my previous change, I got all the subclasses but not zkex itself.

214131

No Perforce job exists for this issue.

0

33007

10 years, 24 weeks, 6 days ago

0|i060gv:

ZooKeeper

ZOOKEEPER-547

Sanity check in QuorumCnxn Manager and quorum communication port.

Bug

Closed

Major

Fixed

Mahadev Konar

08/Oct/09 18:01

26/Mar/10 13:25

17/Nov/09 18:39

3.2.0, 3.2.1

3.2.2, 3.3.0

leaderElection, server

0

2

We need to put some sanity checks in QuorumCnxnManager and the other quorum port for rogue clients. Sometimes a clients might get misconfigured and they might send random characters on such ports. We need to make sure that such rogue clients do not bring down the clients and need to put in some sanity checks with respect to packet lengths and deserialization.

47739

No Perforce job exists for this issue.

4

33008

23/Oct/09 6:44 PM

10 years, 19 weeks, 1 day ago

Reviewed

0|i060h3:

ZooKeeper

ZOOKEEPER-546

add "diskless" ensemble support

New Feature

Resolved

Major

Duplicate

Unassigned

Patrick D. Hunt

08/Oct/09 12:40

18/Nov/11 20:18

06/Sep/10 03:43

server

0

In some cases there is no need to have the ZK data persisted to disk. For example if all you are doing is group membership and leadership
election the data is totally ephemeral, storing on disk is unnecessary. We've also seen cases where any non-ephemeral data can be
easily recovered (say configuration data that's generated/read and loaded into zk) and there is less need to worry about recovery of the
data in the case of catastrophic failure (meaning _all_ replicas are lost, remember, recovery is automatic if 2n+1 servers and <= n servers
fail, even if > n fail manual recovery is still possible as long as at least 1 replica, or replica backup can be recovered)

In these cases it makes sense to have a "diskless" zookeeper ensemble. The result should be improved write performance
an less moving parts (no disk to fail!), simplifiying ops in cases where this can be applied.

214130

No Perforce job exists for this issue.

0

33444

06/Sep/10 3:43 AM

9 years, 29 weeks, 3 days ago

0|i0635z:

ZooKeeper

ZOOKEEPER-545

investigate use of realtime gc as the recommened default for server vm

Improvement

Open

Critical

Unresolved

Unassigned

Patrick D. Hunt

08/Oct/09 12:32

14/Dec/19 06:09

3.7.0

server

2

9

We currently don't recommend that ppl use the realtime gc when running the server, we probably should.

Before we do so we need to verify that it works.
We should make it the default for all our tests.
concurrent vs "g2" or whatever it's called (new in 1.6_15 or something?)
Update all scripts to specify this option
update documentation to include this option and add section in the dev/ops docs detailing it's benefits (in particular latency effects of gc)

Also, -server option? any benefit for us to recommend this as well?

newbie

46278

No Perforce job exists for this issue.

0

68457

28/Oct/09 4:30 PM

5 years, 37 weeks, 3 days ago

0|i0c35z:

ZooKeeper

ZOOKEEPER-544

improve client testability - allow test client to access connected server location

Improvement

Closed

Major

Fixed

Patrick D. Hunt

08/Oct/09 12:29

26/Mar/10 13:25

05/Jan/10 12:32

3.3.0

c client, java client, tests

0

1

This came up recently on the user list. If you are developing tests for your zk client you need to be able to access the server that your
session is currently connected to. The reason is that your test needs to know which server in the quorum to shutdown in order to
verify you are handling failover correctly. Similar for session expiration testing.

however we should be careful, we prefer not to expose this to all clients, this is an implementation detail that we typically
want to hide.

also we should provide this in both the c and java clients

I suspect we should add a protected method on ZooKeeper. This will make a higher bar (user will have to subclass) for
the user to access this method. In tests it's fine, typically you want a "TestableZooKeeper" class anyway. In c we unfortunately
have less options, we can just rely on docs for now.

In both cases (c/java) we need to be very very clear in the docs that this is for testing only and to clearly define semantics.

We should add the following at the same time:

toString() method to ZooKeeper which includes server ip/port, client port, any other information deemed useful (connection stats like send/recv?)

the java ZooKeeper is missing "deterministic connection order" that the c client has. this is also useful for testing. again, protected and
clear docs that this is for testing purposes only!

Any other things we should expose?

47740

No Perforce job exists for this issue.

3

33445

09/Nov/09 2:00 AM

10 years, 12 weeks, 1 day ago

Reviewed

0|i06367:

ZooKeeper

ZOOKEEPER-543

Tests for ZooKeeper examples

Improvement

Closed

Minor

Fixed

Steven Cheng

06/Oct/09 16:32

26/Mar/10 13:25

08/Mar/10 21:09

3.3.0

tests

0

1

Initial attempt to create ZooKeeper tests based on the example code on the website.
Current plan is to test features used in examples using ZooKeeper calls directly. Another approach would be to make more usable abstractions such as those in src/recipes and test those.

47741

No Perforce job exists for this issue.

4

33446

15/Jan/10 5:48 PM

10 years, 3 weeks, 2 days ago

Reviewed

0|i0636f:

ZooKeeper

ZOOKEEPER-542

c-client can spin when server unresponsive

Bug

Closed

Major

Fixed

Christian Wiedmann

05/Oct/09 21:54

26/Mar/10 13:25

07/Oct/09 14:11

3.2.0, 3.2.1

3.3.0

c client

0

2

Due to a mismatch between zookeeper_interest() and zookeeper_process(), when the zookeeper server is unresponsive the client can spin when reconnecting to the server.

In particular, zookeeper_interest() adds ZOOKEEPER_WRITE whenever there is data to be sent, but flush_send_queue() only writes the data if the state is ZOO_CONNECTED_STATE. When in ZOO_ASSOCIATING_STATE, this results in spinning.

This probably doesn't affect production, but I had a runaway process in a development deployment that caused performance issues on the node. This is easy to reproduce in a single node environment by doing a kill -STOP on the server and waiting for the session timeout.

Patch to be added.

47742

No Perforce job exists for this issue.

2

33009

06/Oct/09 12:51 PM

10 years, 25 weeks ago

Reviewed

0|i060hb:

ZooKeeper

ZOOKEEPER-541

zkpython limited to 256 handles

Bug

Closed

Major

Fixed

Henry Robinson

Patrick D. Hunt

05/Oct/09 20:04

26/Mar/10 13:25

13/Oct/09 19:14

3.2.1

3.2.2, 3.3.0

contrib-bindings

0

2

ZOOKEEPER-540

zkpython is currently limited to a max of 256 total handles - not 256 open handles, but rather 256 total handles created
over the lifetime of the python application.

In general this isn't a real issue, however in the case of a long lived application which polls the cluster periodically (closing
the session btw calls) this is an issue.

it would be great if the slots could be reused? or perhaps a more complex structure, such as a linked list, which would allow
dynamic growth/shrinkage of the handle list.

Also see ZOOKEEPER-540

47743

No Perforce job exists for this issue.

1

33010

06/Oct/09 6:01 PM

10 years, 24 weeks, 1 day ago

Reviewed

0|i060hj:

ZooKeeper

ZOOKEEPER-540

zkpython needs better tracking of handle validity

Bug

Closed

Major

Fixed

Henry Robinson

Patrick D. Hunt

01/Oct/09 18:02

26/Mar/10 13:25

08/Oct/09 20:43

3.2.1

3.2.2, 3.3.0

contrib-bindings

0

2

ZOOKEEPER-541

I was getting a python segfault in one of my scripts. Turns out I was closing a session handle and then reusing it (async call). This was causing python to segfault.

zkpython should track handle state and complain, rather than crash, if the handle is invalid (closed).

47744

No Perforce job exists for this issue.

0

33011

08/Oct/09 8:43 PM

10 years, 24 weeks, 6 days ago

0|i060hr:

ZooKeeper

ZOOKEEPER-539

generate eclipse project via ant target

New Feature

Closed

Critical

Fixed

Patrick D. Hunt

01/Oct/09 16:54

26/Mar/10 13:25

05/Oct/09 18:38

3.3.0

build

0

1

Enable eclipse project generation via ant target.

47745

No Perforce job exists for this issue.

1

33447

01/Oct/09 5:14 PM

10 years, 25 weeks, 2 days ago

Reviewed

0|i0636n:

ZooKeeper

ZOOKEEPER-538

zookeeper.async causes python to segfault

Bug

Closed

Critical

Fixed

Henry Robinson

Patrick D. Hunt

30/Sep/09 02:00

26/Mar/10 13:25

01/Oct/09 20:26

3.2.1

3.2.2, 3.3.0

contrib-bindings

0

3

Henry, can you take a look at this, am I doing it right?

calling
zookeeper.async(self.handle, path)
causes python to segfault.

see: http://github.com/phunt/zk-smoketest/blob/master/zk-smoketest.py

47746

No Perforce job exists for this issue.

2

33012

30/Sep/09 7:27 AM

10 years, 25 weeks, 6 days ago

Reviewed

0|i060hz:

ZooKeeper

ZOOKEEPER-537

The zookeeper jar includes the java source files

Bug

Closed

Major

Fixed

Thomas Dudziak

25/Sep/09 18:25

26/Mar/10 13:25

06/Nov/09 16:20

3.3.0

build

1

3

ZOOKEEPER-95, ZOOKEEPER-224

This is a problem if you use zookeeper as a dependency in maven because for whatever reason the maven compiler plugin will pick up the java files in the jar and compile them to the output directory. From there they will land in the generated jar file for whatever project happens to depend on zookeeper thus introducing duplicate classes (once in zookeeper.jar, once in the project's artifact).

47747

No Perforce job exists for this issue.

2

33013

30/Sep/09 1:13 PM

10 years, 20 weeks, 4 days ago

Reviewed

0|i060i7:

ZooKeeper

ZOOKEEPER-536

the initial size of the hashsets for the watcher is too large

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

25/Sep/09 17:56

10/Oct/13 13:33

0

1

setting a watches on a lot of different nodes can be expensive if there is only one watch set on each node. by default the hashset we use to track watches has 16 entries and takes up about 160 bytes. we should probably make the initial size much lower.

newbie

214129

No Perforce job exists for this issue.

0

68458

27/Sep/09 4:06 PM

10 years, 26 weeks, 4 days ago

0|i0c367:

ZooKeeper

ZOOKEEPER-535

ivy task does not enjoy being defined twice (build error)

Bug

Closed

Major

Fixed

Patrick D. Hunt

24/Sep/09 13:56

26/Mar/10 13:25

24/Sep/09 14:52

3.3.0

build

0

2

Ant resolves the same dependencies multiple times if multiple targets are run on the command line:

"ant b c", where b and c both depend on a, results in a being executed twice. However if you have a
target d which depends on both b and c, "ant d" will only result in a being executed once.

say "ant jar compile-test" is run, this will currently fail as ivy-init is run twice, resulting in the taskdef failing.

Rather we need a guard on the ivy-init target itself to ensure it isn't run twice.

ie: put an unless attrib on ivy-init, then set that property in the body of the target

47748

No Perforce job exists for this issue.

1

33014

24/Sep/09 2:52 PM

10 years, 27 weeks ago

Reviewed

0|i060if:

ZooKeeper

ZOOKEEPER-534

The test target in contib/bookkeeper does not depend on jar target.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

23/Sep/09 14:46

01/May/13 22:29

16/Dec/09 18:46

3.2.0, 3.2.1

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-623

The test target in contib/bookkeeper does not depend on jar target. So the ant test target gives compilation errors if the main is not compiled which can be prevented if it depends on jar. It can then check if main has been compiled or not and throw out a reasonable error.

47749

No Perforce job exists for this issue.

1

33015

02/Dec/09 3:46 AM

10 years, 15 weeks ago

Reviewed

0|i060in:

ZooKeeper

ZOOKEEPER-533

ant error running clean twice

Bug

Closed

Major

Fixed

Patrick D. Hunt

23/Sep/09 13:41

26/Mar/10 13:25

23/Sep/09 17:54

3.3.0

build

0

2

if clean is run twice in a row (ie already clean, then run clean) an error is generated.

47750

No Perforce job exists for this issue.

2

33016

23/Sep/09 4:00 PM

10 years, 27 weeks ago

Reviewed

0|i060iv:

ZooKeeper

ZOOKEEPER-532

java compiler should be target Java 1.5

Bug

Closed

Major

Fixed

Hiram R. Chirino

22/Sep/09 16:09

26/Mar/10 13:25

18/Nov/09 12:30

3.2.1

3.3.0

0

2

The jars released in 3.2.1 will not run on Java 1.5. With a small build change, it is possible to generate jars that will run on Java 1.5.

47751

No Perforce job exists for this issue.

2

33017

22/Sep/09 4:38 PM

10 years, 18 weeks, 6 days ago

Reviewed

0|i060j3:

ZooKeeper

ZOOKEEPER-531

Hudson trunk failure in heirarchical quorum test (interrupt problem)

Bug

Closed

Blocker

Cannot Reproduce

Flavio Paiva Junqueira

Patrick D. Hunt

21/Sep/09 13:09

26/Mar/10 13:25

12/Jan/10 15:12

3.3.0

quorum, server

0

1

Flavio, can you take a look, this is very unusual. This test seems to be failing due to interrupt being received and not handled properly:

http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/470/testReport/org.apache.zookeeper.test/HierarchicalQuorumTest/testHierarchicalQuorum/

I don't know why the interrupt would be received though...

here's an example:
2009-09-21 10:46:47,681 - WARN [Thread-8:QuorumCnxManager$SendWorker@539] - Interrupted while waiting for message on queue
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1899)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1934)
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:533)

214128

No Perforce job exists for this issue.

0

33018

10 years, 27 weeks, 3 days ago

0|i060jb:

ZooKeeper

ZOOKEEPER-530

Memory corruption: Zookeeper c client IPv6 implementation does not honor struct sockaddr_in6 size

Bug

Closed

Major

Fixed

Isabel Drost-Fromm

21/Sep/09 05:17

26/Mar/10 13:25

21/Oct/09 17:01

3.2.0, 3.2.1

3.3.0

c client

0

3

I tried to run zookeeper c-client on a machine with IPv6 enabled. When connecting to the IPv6 address a connect(...) gave a "Address family not supported by protocol" error. The reason was, that a few lines earlier, the socket was opened with PF_INET instead of PF_INET6. Changing that the following way:

{code}
if (zh->addrs[zh->connect_index].sa_family == AF_INET) {
zh->fd = socket(PF_INET, SOCK_STREAM, 0);
} else {
zh->fd = socket(PF_INET6, SOCK_STREAM, 0);
}
{code}

turned the error message into "Invalid argument".

When printing out sizeof(struct sockaddr), sizeof(struct sockaddr_in) and sizeof(struct sockaddr_in6) I got sockaddr: 16, sockaddr_in: 16 and sockaddr_in6: 28.

So in the code calling

{code}
connect(zh->fd, &zh->addrs[zh->connect_index], sizeof(struct sockaddr_in));
{code}

the parameter address_len is too small.

Same applies to how IPv6 addresses are handled in the function getaddrs(zhandle_t *zh).

(Big Thanks+kiss to Thilo Fromm for helping me debug this.)

47752

No Perforce job exists for this issue.

3

33019

21/Sep/09 12:47 PM

10 years, 23 weeks ago

get the c client working with IPV6.

Reviewed

ipv6

0|i060jj:

ZooKeeper

ZOOKEEPER-529

Use Ivy to pull dependencies and also generate pom

Improvement

Closed

Major

Fixed

Patrick D. Hunt

18/Sep/09 18:23

26/Mar/10 13:25

22/Sep/09 19:00

3.3.0

build

0

1

ZOOKEEPER-224

Integrate Ivy into the build. Generate Pom and signed jars to allow zk jars to be officially deployed to a maven repo.

47753

No Perforce job exists for this issue.

3

33448

18/Sep/09 7:05 PM

10 years, 27 weeks ago

using ivy for the build. Pulling all the dependent jars using ivy.

Reviewed

0|i0636v:

ZooKeeper

ZOOKEEPER-528

c client exists() call with watch on large number of nodes (>100k) causes connection loss

Bug

Closed

Critical

Invalid

Patrick D. Hunt

16/Sep/09 14:46

26/Mar/10 13:25

14/Oct/09 13:27

3.2.0

3.3.0

c client

0

1

If I create 100k nodes on /misc then

CPPUNIT_ASSERT_EQUAL(0, zoo_get_children(zh2, "/misc", 0, &children));
for (int i = 0; i < children.count; i++) {
sprintf(path, "/misc/%s", children.data[i]);
CPPUNIT_ASSERT_EQUAL(0, zoo_exists(zh2, path, 1, &stat));
CPPUNIT_ASSERT_EQUAL(0, zoo_wexists(zh3, path, watcher, &ctx3, &stat));
}

around 47k or so through the loop the client fails with -4 (connection loss), the client timeout is 30 seconds. The server command port shows the following, so it looks like it's not the server but some issue with watcher reg on the c client?

phunt@valhalla:~$ echo stat | nc localhost 22181
Zookeeper version: 3.3.0--1, built on 07/22/2009 23:55 GMT
Clients:
/127.0.0.1:45729[1](queued=0,recved=100024,sent=0)
/127.0.0.1:50229[1](queued=0,recved=0,sent=0)
/127.0.0.1:45731[1](queued=0,recved=47116,sent=0)
/127.0.0.1:45730[1](queued=0,recved=47117,sent=1)

Latency min/avg/max: 0/196/1026
Received: 194257
Sent: 1
Outstanding: 0
Zxid: 0x186a4
Mode: standalone
Node count: 100005

729 is a separate client - the one that created the nodes originally.

731 and 730 are zh2/zh3 in the code.

214127

No Perforce job exists for this issue.

0

33020

10 years, 24 weeks, 1 day ago

workaround: the test environment in this case had a max heap of 64m, by increasing the max mem via -Xmx the performance issue was addressed and the test ran fine.

0|i060jr:

ZooKeeper

ZOOKEEPER-527

hudson trunk failure in quota test

Bug

Closed

Critical

Cannot Reproduce

Mahadev Konar

Patrick D. Hunt

16/Sep/09 13:34

26/Mar/10 13:25

07/Dec/09 16:13

3.2.1

3.3.0

server

0

1

the following test failed:

http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/462/testReport/org.apache.zookeeper.test/QuorumQuotaTest/testQuotaWithQuorum/

here's the interesting log:

2009-09-16 10:35:52,728 - WARN [CommitProcessor:1:DataTree@409] - Quota exceeded: /a bytes=1808 limit=1000
2009-09-16 10:36:34,000 - INFO [SessionTracker:SessionTrackerImpl@133] - Expiring session 0x423c26c1d220000
2009-09-16 10:36:12,725 - WARN [main-SendThread(localhost:11225):ClientCnxn$SendThread@969] - Exception closing session 0x423c26c1d220000 to sun.nio.ch.SelectionKeyImpl@a7dd39
java.io.IOException: TIMED OUT
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:908)

looks like a create call hung and the session eventually expired

perhaps updating the quota had some issue? deadlock or ...

214126

No Perforce job exists for this issue.

0

33021

07/Dec/09 4:13 PM

10 years, 16 weeks, 3 days ago

0|i060jz:

ZooKeeper

ZOOKEEPER-526

Change the access pattern in the DataNode from direct access to the use of getters and setters

Improvement

Open

Minor

Unresolved

Unassigned

Erik Holstad

11/Sep/09 19:42

14/Dec/19 06:08

3.7.0

server

0

1

70736

No Perforce job exists for this issue.

0

68459

10 years, 28 weeks, 6 days ago

0|i0c36f:

ZooKeeper

ZOOKEEPER-525

Changing children Set<String> to Set<byte[]> in DataNode

Improvement

Open

Minor

Unresolved

Unassigned

Erik Holstad

11/Sep/09 16:17

14/Dec/19 06:08

3.7.0

server

0

1

For every instance of string there is an overhead of 48B compared to using byte[], on a 64 bit system, that seems unnecessary.

70728

No Perforce job exists for this issue.

0

68460

10 years, 28 weeks, 6 days ago

0|i0c36n:

ZooKeeper

ZOOKEEPER-524

DBSizeTest is not really testing anything

Bug

Closed

Minor

Fixed

Benjamin Reed

Patrick D. Hunt

10/Sep/09 19:36

26/Mar/10 13:25

19/Feb/10 20:21

3.3.0

server, tests

0

2

DBSizeTest looks like it should be testing latency, but it doesn't seem to do it (assert is commented out).

We need to decide if this test should be fixed, or just dropped.

Also note: this test takes 40seconds on my system. Way too long. Perhaps async create operations should be used
to populate the database. I also noticed that data size has a big impact on overall test time (1k vs 5 bytes is something
like a 2x time diff for time to run the test).

47754

No Perforce job exists for this issue.

0

33022

28/Jan/10 12:23 PM

10 years, 5 weeks, 5 days ago

0|i060k7:

ZooKeeper

ZOOKEEPER-523

zookeeper c client should shutdown if it sees its zxid is too high from all the server its connecting to.

Bug

Open

Major

Unresolved

Unassigned

Mahadev Konar

08/Sep/09 13:37

14/Dec/19 06:06

3.7.0

c client, java client, server

1

0

In one of the scenarios, one of our users cleaned up the server database, upgraded the zookeeper servers to 3.* from 2.* and did not shut down there clients. The clients kept spinning since they couldnt find a server that was up to date. Though this was a mistake on the users side, but the spinning of clients caused more problems (like zookeeper server running out of file handles since the clients kept spinning throguh servers). In such a case we should shut down the clients since its this should never happen.

66054

No Perforce job exists for this issue.

0

33023

8 years, 37 weeks ago

0|i060kf:

ZooKeeper

ZOOKEEPER-522

zookeeper client should throttle if its not able to connect to any of the servers.

Improvement

Open

Major

Unresolved

Unassigned

Mahadev Konar

04/Sep/09 16:47

14/Dec/19 06:08

3.2.0

3.7.0

0

2

Currently the zookeeper client library keeps connecting to servers if all of them are unreachable. It will go through the list time and again and try to connect. Sometimes, this might cause problems like too many clients retrying connect to servers (and there might be something wrong/delay with servers) wherein the clients will give up and will try reconnecting to other servers. This causes a huge churn in client connections sometimes leading to the zookeeper server running out of file handles.

71157

No Perforce job exists for this issue.

0

68461

15/Jun/11 4:34 PM

8 years, 41 weeks, 1 day ago

0|i0c36v:

ZooKeeper

ZOOKEEPER-521

include autotools (autoconf/automake) in NOTICE file

Task

Closed

Minor

Invalid

Patrick D. Hunt

04/Sep/09 15:07

26/Mar/10 13:25

16/Sep/09 14:34

3.3.0

build, documentation

0

1

The toplevel notice file should be updated to credit the gnu autotools project, something like:

This project includes scripts generated by GNU autotools (autoconf/automake/etc...).
These files are released under the Apache license.
This is permissible under the license clarification at:
http://ftp.gnu.org/old-gnu/Manuals/autoconf-2.53/html_node/Distributing.html#Distributing

we should also take the opportunity to adjust the existing text as detailed here:
http://apache.org/legal/src-headers.html#notice

214125

No Perforce job exists for this issue.

0

33449

08/Sep/09 5:26 PM

10 years, 28 weeks, 1 day ago

0|i06373:

ZooKeeper

ZOOKEEPER-520

add static/readonly client resident serverless zookeeper

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

03/Sep/09 20:06

15/Jan/10 17:21

c client, java client

0

1

Occasionally people (typically ops) has asked for the ability to start a ZK client with a hardcoded, local, non cluster based session. Meaning that you can bring up a particular client with a hardcoded/readonly view of the ZK namespace even if the zk cluster is not available. This seems useful for a few reasons:

1) unforseen problems - a client might be brought up and partial application service restored even in the face of catastrophic cluster failure

2) testing - client could be brought up with a hardcoded configuration for testing purposes. we might even be able to extend this idea over time to allow "simulated changes" ie - simulate other clients making changes in the namespace, perhaps simulate changes in the state of the cluster (testing state change is often hard for users of the client interface)

Seems like this shouldn't be too hard for us to add. The session could be established with a URI for a local/remote file rather than a URI of the cluster servers. The client would essentially read this file which would be a simple representation of the znode namespace.

/foo/bar "abc"
/foo/bar2 "def"
etc...

In the pure client readonly case this is simple. We might also want to allow writes to the namespace (essentially back this with an in memory hash) for things like group membership (so that the client continues to function).

Obv this wouldn't work in some cases, but it might work in many and would allow further options for users wrt building a relable/recoverable service on top of ZK.

214124

No Perforce job exists for this issue.

0

68462

10 years, 30 weeks ago

0|i0c373:

ZooKeeper

ZOOKEEPER-519

Followerhandler should close the socket if it gets an exception on a write.

Bug

Closed

Major

Fixed

Mahadev Konar

25/Aug/09 19:49

26/Mar/10 13:25

18/Nov/09 12:45

3.3.0

0

2

We noticed this in our tests -

{code}

java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at org.apache.jute.BinaryOutputArchive.writeBuffer(BinaryOutputArchive.java:122)
at org.apache.zookeeper.server.DataNode.serialize(DataNode.java:126)
at org.apache.jute.BinaryOutputArchive.writeRecord(BinaryOutputArchive.java:126)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:878)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:890)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:890)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:890)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:890)
at org.apache.zookeeper.server.DataTree.serializeNode(DataTree.java:890)
at org.apache.zookeeper.server.DataTree.serialize(DataTree.java:940)
at org.apache.zookeeper.server.util.SerializeUtils.serializeSnapshot(SerializeUtils.java:102)
at org.apache.zookeeper.server.ZooKeeperServer.serializeSnapshot(ZooKeeperServer.java:269)
at org.apache.zookeeper.server.quorum.FollowerHandler.run(FollowerHandler.java:263)
{code}

So the followerhandler got an exception while writing to the socket but the follower was still waiting on the socket for a read and got a read timeout after 60 seconds or so. To just make sure we handle this rightly, we should close the socket at the followerhandler when we get an excpetion, so that the follower immediately recognizes that its disconnected from the leader.

47755

No Perforce job exists for this issue.

1

33024

02/Nov/09 5:24 PM

10 years, 18 weeks, 6 days ago

Reviewed

0|i060kn:

ZooKeeper

ZOOKEEPER-518

DEBUG message for outstanding proposals in leader should be moved to trace.

Test

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

25/Aug/09 19:38

07/May/10 13:40

27/Aug/09 17:58

4.0.0

3.2.1, 3.3.0

tests

0

1

Non specific

this is the code in Leader.java
{code}
if (LOG.isDebugEnabled()) {
LOG.debug("Ack zxid: 0x" + Long.toHexString(zxid));
for (Proposal p : outstandingProposals.values()) {
long packetZxid = p.packet.getZxid();
LOG.debug("outstanding proposal: 0x"
+ Long.toHexString(packetZxid));
}
LOG.debug("outstanding proposals all");
}

{code}

We should move this debug to trace since it will cause really high latencies in response times from zookeeper servers in case folks want to use DEBUG logging for servers.

logging

47756

No Perforce job exists for this issue.

1

33450

26/Aug/09 1:04 PM

10 years, 30 weeks, 6 days ago

The released of this module should be done only in apache org as this module is move to the debug to trace since it will cause really high latenciesinin response times from zookeeper services in case folks want to use DEBUGlogging for servers.

Reviewed

0|i0637b:

ZooKeeper

ZOOKEEPER-517

NIO factory fails to close connections when the number of file handles run out.

Bug

Resolved

Critical

Duplicate

Jay Shrauner

Mahadev Konar

25/Aug/09 19:32

27/Jul/12 01:18

3.4.3, 3.5.0

server

1

2

ZOOKEEPER-801

The code in NIO factory is such that if we fail to accept a connection due to some reasons (too many file handles maybe one of them) we do not close the connections that are in CLOSE_WAIT. We need to call an explicit close on these sockets and then close them. One of the solutions might be to move doIO before accpet so that we can still close connection even if we cannot accept connections.

71225

No Perforce job exists for this issue.

1

33025

01/Jul/10 5:19 PM

7 years, 38 weeks ago

Fix NIO accept loop error handling

0|i060kv:

ZooKeeper

ZOOKEEPER-516

add support for 10 minute test ie "pre-commit" test

Task

Closed

Minor

Fixed

Patrick D. Hunt

25/Aug/09 19:27

26/Mar/10 13:25

15/Sep/09 20:10

3.3.0

tests

0

1

Our unit tests are now taking > 10 minutes (20 actually). We need to set things up such that we have a < 10 minute "pre commit" build, and a longer running test of tests (unbounded for all intents/purposes). Mainly this requires us to categorize our tests.

47757

No Perforce job exists for this issue.

2

33451

11/Sep/09 7:53 PM

10 years, 28 weeks, 2 days ago

Reviewed

0|i0637j:

ZooKeeper

ZOOKEEPER-515

Zookeeper quorum didn't provide service when restart after an "Out of memory" crash

Bug

Resolved

Major

Invalid

Unassigned

Qian Ye

25/Aug/09 09:10

06/Sep/11 12:01

3.2.0

3.5.0

server

0

Linux 2.6.9-52bs-4core #2 SMP Wed Jan 16 14:44:08 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
Jdk: 1.6.0_14

The Zookeeper quorum, containing 5 servers, didn't provide service when restart after an "Out of memory" crash.

It happened as following:
1. we built a Zookeeper quorum which contained 5 servers, say 1, 3, 4, 5, 6 (have no 2), and 6 was the leader.
2. we created 18 threads on 6 different servers to set and get data from a znode in the Zookeeper at the same time. The size of the data is 1MB. The test threads did their job as fast as possible, no pause between two operation, and they repeated the setting and getting 4000 times.
3. the Zookeeper leader crashed about 10 mins after the test threads started. The leader printed out the log:

2009-08-25 12:00:12,301 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x523
4223c2dc00b5 due to java.io.IOException: Read error
2009-08-25 12:00:12,318 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x523
4223c2dc00b6 due to java.io.IOException: Read error
2009-08-25 12:03:44,086 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x523
4223c2dc00b8 due to java.io.IOException: Read error
2009-08-25 12:04:53,757 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x523
4223c2dc00b7 due to java.io.IOException: Read error
2009-08-25 12:15:45,151 - FATAL [SyncThread:0:SyncRequestProcessor@131] - Severe unrecoverable error, exiting
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:180)
at org.apache.jute.BinaryOutputArchive.writeInt(BinaryOutputArchive.java:55)
at org.apache.zookeeper.txn.SetDataTxn.serialize(SetDataTxn.java:42)
at org.apache.zookeeper.server.persistence.Util.marshallTxnEntry(Util.java:262)
at org.apache.zookeeper.server.persistence.FileTxnLog.append(FileTxnLog.java:154)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.append(FileTxnSnapLog.java:268)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:100)

It is clear that the leader ran out of memory. then the server 4 was down almost at the same time, and printed out the log:
2009-08-25 12:15:45,995 - ERROR [FollowerRequestProcessor:3:FollowerRequestProcessor@91] - Unexpected exception causing
exit
java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at java.io.FilterOutputStream.write(FilterOutputStream.java:80)
at org.apache.jute.BinaryOutputArchive.writeBuffer(BinaryOutputArchive.java:119)
at org.apache.zookeeper.server.quorum.QuorumPacket.serialize(QuorumPacket.java:51)
at org.apache.jute.BinaryOutputArchive.writeRecord(BinaryOutputArchive.java:123)
at org.apache.zookeeper.server.quorum.Follower.writePacket(Follower.java:97)
at org.apache.zookeeper.server.quorum.Follower.request(Follower.java:399)
at org.apache.zookeeper.server.quorum.FollowerRequestProcessor.run(FollowerRequestProcessor.java:86)
2009-08-25 12:15:45,996 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x423
4ab894330075 due to java.net.SocketException: Broken pipe
2009-08-25 12:15:45,996 - FATAL [SyncThread:3:SyncRequestProcessor@131] - Severe unrecoverable error, exiting
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.zookeeper.server.quorum.Follower.writePacket(Follower.java:100)
at org.apache.zookeeper.server.quorum.SendAckRequestProcessor.flush(SendAckRequestProcessor.java:52)
at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:147)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:92)
2009-08-25 12:15:45,995 - WARN [QuorumPeer:/0.0.0.0:2181:Follower@309] - Exception when following the leader
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.zookeeper.server.quorum.Follower.writePacket(Follower.java:100)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:256)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-08-25 12:15:46,022 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 12:15:46,022 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 12:15:46,023 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running

It was really strange that when the 2 server down, the other three servers cannot provide service any more, the 'stat' command all leaded to "ZooKeeperServer not running".

4. I restarted the server 6(the former leader) and the server 4. But the service didn't come back. All the five servers printed "ZookeeperServer not running". The server 6 printed the logs:

2009-08-25 14:02:15,395 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:02:27,703 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: Responded to info probe
2009-08-25 14:02:28,733 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:02:42,070 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:02:55,407 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:03:08,744 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:03:22,080 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:03:29,396 - ERROR [main:Util@238] - Last transaction was partial.
2009-08-25 14:03:35,417 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:03:48,761 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running

The server 4 printed logs like:

2009-08-25 14:03:48,747 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:04:02,091 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:04:15,427 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:04:17,816 - WARN [QuorumPeer:/0.0.0.0:2181:Follower@164] - Unexpected exception, tries=0
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:525)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-08-25 14:04:18,820 - WARN [QuorumPeer:/0.0.0.0:2181:Follower@164] - Unexpected exception, tries=1
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:525)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-08-25 14:04:19,823 - WARN [QuorumPeer:/0.0.0.0:2181:Follower@164] - Unexpected exception, tries=2
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:525)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-08-25 14:04:28,764 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:04:42,101 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running

the server 1, 3, 5 printed out the logs like:

2009-08-25 14:01:35,396 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: ZooKeeperServer not running
2009-08-25 14:01:36,554 - WARN [QuorumPeer:/0.0.0.0:2181:LeaderElection@194] - Ignoring exception while looking for lea
der
java.net.SocketTimeoutException: Receive timed out
at java.net.PlainDatagramSocketImpl.receive0(Native Method)
at java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
at java.net.DatagramSocket.receive(DatagramSocket.java:712)
at org.apache.zookeeper.server.quorum.LeaderElection.lookForLeader(LeaderElection.java:170)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
2009-08-25 14:01:37,758 - WARN [QuorumPeer:/0.0.0.0:2181:LeaderElection@194] - Ignoring exception while looking for lea
der
java.net.SocketTimeoutException: Receive timed out
at java.net.PlainDatagramSocketImpl.receive0(Native Method)
at java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136)
at java.net.DatagramSocket.receive(DatagramSocket.java:712)
at org.apache.zookeeper.server.quorum.LeaderElection.lookForLeader(LeaderElection.java:170)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:488)
2009-08-25 14:01:37,865 - WARN [QuorumPeer:/0.0.0.0:2181:Follower@164] - Unexpected exception, tries=0
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:525)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:156)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-08-25 14:01:38,289 - WARN [NIOServerCxn.Factory:2181:NIOServerCnxn@497] - Exception causing close of session 0x0 d
ue to java.io.IOException: Responded to info probe

my zoo.cfg is like:
tickTime=2000
dataDir=./status/
clientPort=2181
initLimit=10
syncLimit=2
server.1=10.81.11.107:2888:3888
server.2=10.81.11.106:2888:3888
server.3=10.81.11.89:2888:3888
server.4=10.81.11.99:2888:3888
server.5=10.81.11.79:2888:3888

Several questions:
1. Why the leader selection failed after the restart?
2. Is the size of data too big to be processed properly?
3. How can I recover from this situation? Can I just remove the version-2 directory on server 6(the former leader) and restart the server?

62189

No Perforce job exists for this issue.

0

33026

25/Aug/09 2:10 PM

8 years, 29 weeks, 2 days ago

0|i060l3:

ZooKeeper

ZOOKEEPER-514

test failure on trunk in testCnxManager - NPE

Bug

Closed

Major

Won't Fix

Unassigned

Patrick D. Hunt

24/Aug/09 13:44

05/Sep/09 18:36

24/Aug/09 15:04

3.2.0

3.2.1, 3.3.0

0

1

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/424/testReport/org.apache.zookeeper.test/CnxManagerTest/testCnxManager/

Flavio can you take a look?

214123

No Perforce job exists for this issue.

0

33027

10 years, 31 weeks, 3 days ago

0|i060lb:

ZooKeeper

ZOOKEEPER-513

C client disconnect with stand-alone server abnormally

Bug

Resolved

Major

Cannot Reproduce

Unassigned

Qian Ye

22/Aug/09 08:12

10/Oct/13 13:33

3.2.0

3.5.0

c client

0

Linux 2.6.9-52bs #2 SMP Fri Jan 26 13:34:38 CST 2007 x86_64 x86_64 x86_64 GNU/Linux
Jdk: 1.6.0_14

The client which created an ephemeral node at the zookeeper server, printed the following log

WARNING: 08-20 03:09:20: auto * 182894118176 [logid:][reqip:][auto_exchanger_zk_basic.cpp:605]get children fail.[/forum/elect_nodes][-7][operation timeout]

and the Zookeeper client printed the following log (the log level is INFO)

2009-08-19 21:36:18,067:3813(0x9556c520):ZOO_INFO@log_env@545: Client environment:zookeeper.version=zookeeper C client 3.2.0
606 2009-08-19 21:36:18,067:3813(0x9556c520):ZOO_INFO@log_env@549: Client environment:host.name=jx-ziyuan-test00.jx.baidu.com
607 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@557: Client environments.name=Linux
608 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@558: Client environments.arch=2.6.9-52bs
609 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@559: Client environments.version=#2 SMP Fri Jan 26 13:34:38 CST 2007
610 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@567: Client environment:user.name=club
611 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@577: Client environment:user.home=/home/club
612 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@log_env@589: Client environment:user.dir=/home/club/user/luhongbo/auto-exchanger
613 2009-08-19 21:36:18,068:3813(0x9556c520):ZOO_INFO@zookeeper_init@613: Initiating client connection, host=127.0.0.1:2181,127.0.0.1:2182 sessionTimeout=2000 wa tcher=0x408c56 sessionId=0x0 sessionPasswd=<null> context=(nil) flags=0
614 2009-08-19 21:36:18,069:3813(0x41401960):ZOO_INFO@check_events@1439: initiated connection to server [127.0.0.1:2181]
615 2009-08-19 21:36:18,070:3813(0x41401960):ZOO_INFO@check_events@1484: connected to server [127.0.0.1:2181] with session id=1232c1688a20093
616 2009-08-20 02:48:01,780:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 520ms
617 2009-08-20 03:08:52,332:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 14ms
618 2009-08-20 03:09:04,666:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 48ms
619 2009-08-20 03:09:09,733:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 24ms
620 2009-08-20 03:09:20,289:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 264ms
621 2009-08-20 03:09:20,295:3813(0x41401960):ZOO_ERROR@handle_socket_error_msg@1388: Socket [127.0.0.1:2181] zk retcode=-7, errno=110(Connection timed out): conn ection timed out (exceeded timeout by 264ms)
622 2009-08-20 03:09:20,309:3813(0x41401960):ZOO_WARN@zookeeper_interest@1335: Exceeded deadline by 284ms
623 2009-08-20 03:09:20,309:3813(0x41401960):ZOO_ERROR@handle_socket_error_msg@1433: Socket [127.0.0.1:2182] zk retcode=-4, errno=111(Connection refused): server refused to accept the client
624 2009-08-20 03:09:20,353:3813(0x41401960):ZOO_INFO@check_events@1439: initiated connection to server [127.0.0.1:2181]
625 2009-08-20 03:09:20,552:3813(0x41401960):ZOO_INFO@check_events@1484: connected to server [127.0.0.1:2181] with session id=1232c1688a20093

The problem happened at 03:09:20, it seems that the zookeeper refused to accept the client, and I don't know why.

the zoo.cfg is like:
# The number of milliseconds of each tick
tickTime=500
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=./data/
# the port at which the clients will connect
clientPort=2181

the C client used multi-thread library, and the session timeout is set to 2000 when the zookeeper handler was initialized.

70784

No Perforce job exists for this issue.

0

33028

24/Aug/09 2:01 PM

10 years, 3 weeks, 6 days ago

0|i060lj:

ZooKeeper

ZOOKEEPER-512

FLE election fails to elect leader

Bug

Closed

Blocker

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

19/Aug/09 20:11

26/Mar/10 13:25

27/Oct/09 17:14

3.2.0

3.3.0

quorum, server

0

2

I was doing some fault injection testing of 3.2.1 with ZOOKEEPER-508 patch applied and noticed that after some time the ensemble failed to re-elect a leader.

See the attached log files - 5 member ensemble. typically 5 is the leader

Notice that after 16:23:50,525 no quorum is formed, even after 20 minutes elapses w/no quorum

environment:

I was doing fault injection testing using aspectj. The faults are injected into socketchannel read/write, I throw exceptions randomly at a 1/200 ratio (rand.nextFloat() <= .005 => throw IOException

You can see when a fault is injected in the log via:
2009-08-19 16:57:09,568 - INFO [Thread-74:ReadRequestFailsIntermittently@38] - READPACKET FORCED FAIL

vs a read/write that didn't force fail:
2009-08-19 16:57:09,568 - INFO [Thread-74:ReadRequestFailsIntermittently@41] - READPACKET OK

otw standard code/config (straight fle quorum with 5 members)

also see the attached jstack trace. this is for one of the servers. Notice in particular that the number of sendworkers != the number of recv workers.

47758

No Perforce job exists for this issue.

10

33029

20/Aug/09 6:18 AM

10 years, 22 weeks, 1 day ago

Reviewed

0|i060lr:

ZooKeeper

ZOOKEEPER-511

bad error handling in FollowerHandler.sendPackets

Bug

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

19/Aug/09 18:35

26/Mar/10 13:25

11/Mar/10 09:06

3.2.0

3.3.0

quorum, server

0

2

in FollowerHandler if sendPackets gets an ioexception on writeRecord the send thread will exit, however the
socket isn't necessarily closed.

2009-08-19 15:28:46,869 - WARN [Sender-/127.0.0.1:58179:FollowerHandler@131] - Unexpected exception
at org.apache.zookeeper.server.quorum.FollowerHandler.sendPackets(FollowerHandler.java:128)
at org.apache.zookeeper.server.quorum.FollowerHandler.access$0(FollowerHandler.java:107)
at org.apache.zookeeper.server.quorum.FollowerHandler$1.run(FollowerHandler.java:325)

This results in the follower taking a very long time to recover and rejoin the quorum.

47759

No Perforce job exists for this issue.

1

33030

05/Mar/10 6:08 PM

10 years, 2 weeks, 6 days ago

0|i060lz:

ZooKeeper

ZOOKEEPER-510

zkpython lumps all exceptions as IOError, needs specialized exceptions for KeeperException types

Bug

Closed

Major

Fixed

Henry Robinson

Patrick D. Hunt

19/Aug/09 17:13

26/Mar/10 13:25

13/Oct/09 20:14

3.2.0

3.2.2, 3.3.0

contrib-bindings

0

2

The current zkpython bindings always throw "IOError("text")" exceptions, even for ZK specific exceptions such as NODEEXISTS. This makes it difficult (error prone) to handle exceptions in python code. You can't easily pickup a connection loss vs a node exists for example. Of course you could match the error string, but this seems like a bad idea imo.

We need to add specific exception types to the python binding that map directly to KeeperException/java types. It would also be useful to include the information provided by the KeeperException (like path in some cases), etc... as part of the error thrown to the python code. Would probably be a good idea to stay as close to java api as possible wrt mapping the errors.

47760

No Perforce job exists for this issue.

5

33031

23/Sep/09 2:12 PM

10 years, 24 weeks, 1 day ago

Reviewed

0|i060m7:

ZooKeeper

ZOOKEEPER-509

Truncating of logs does not work right at the followers.

Bug

Closed

Blocker

Fixed

Benjamin Reed

Mahadev Konar

19/Aug/09 13:41

05/Sep/09 18:36

25/Aug/09 01:57

3.2.1, 3.3.0

0

2

ZOOKEEPER-508, ZOOKEEPER-483

The truncating of logs does not work right because we use BufferedInputStream and the truncation happens at the buffered read which is greater than the actual truncate position.

47761

No Perforce job exists for this issue.

0

33032

10 years, 31 weeks, 2 days ago

fixed in ZOOKEEPER-508.

0|i060mf:

ZooKeeper

ZOOKEEPER-508

proposals and commits for DIFF and Truncate messages from the leader to followers is buggy.

Bug

Closed

Blocker

Fixed

Mahadev Konar

17/Aug/09 14:39

05/Sep/09 18:36

25/Aug/09 01:56

3.2.1, 3.3.0

quorum

0

2

ZOOKEEPER-509, ZOOKEEPER-483

The proposals and commits sent by the leader after it asks the followers to truncate there logs or starts sending a diff has missing messages which causes out of order commits messages and causes the followers to shutdown because of these out of order commits.

47762

No Perforce job exists for this issue.

6

33033

19/Aug/09 11:55 PM

10 years, 31 weeks, 2 days ago

includes the fix for ZOOKEEPER-483, ZOOKEEPER-509.

Reviewed

0|i060mn:

ZooKeeper

ZOOKEEPER-507

BookKeeper client re-write

Improvement

Closed

Major

Fixed

Utkarsh Srivastava

Flavio Paiva Junqueira

15/Aug/09 04:26

26/Mar/10 13:25

26/Jan/10 18:20

3.3.0

contrib-bookkeeper

0

1

BOOKKEEPER-691, ZOOKEEPER-505, ZOOKEEPER-502, ZOOKEEPER-503, ZOOKEEPER-504, ZOOKEEPER-461, ZOOKEEPER-607

Error handling is far from ideal currently in the BookKeeper client.

47763

No Perforce job exists for this issue.

6

33452

22/Oct/09 5:06 PM

10 years, 9 weeks, 1 day ago

bookkeeper code rewrite with optimization on disk write scheduling and scheduling of ledger writes on the client side.

Reviewed

0|i0637r:

ZooKeeper

ZOOKEEPER-506

QuorumBase should use default leader election

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

15/Aug/09 04:24

26/Mar/10 13:25

15/Dec/09 17:24

3.3.0

leaderElection

0

1

ZOOKEEPER-599

QuorumBase currently does not use the default implementation of leader election.

47764

No Perforce job exists for this issue.

0

33453

15/Dec/09 5:24 PM

10 years, 15 weeks, 1 day ago

0|i0637z:

ZooKeeper

ZOOKEEPER-505

testAsyncCreateClose is badly broken

Bug

Closed

Critical

Fixed

Utkarsh Srivastava

08/Aug/09 16:57

26/Mar/10 13:25

26/Jan/10 18:17

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-507

The test case testAsyncCreateClose is badly broken. I was wondering why all the unit tests are passing inspite of having found so many different problems with LedgerManagementProcessor.

There is a big try-catch block sitting in the test case that catches all exception, prints their stack trace, and exits, thereby allowing the test to pass. In general, unit tests shouldnt catch exceptions unless it is something you are expecting that will happen.

Another problem is that the same ControlObject is used for synchronization throughout. Since we already have the problem of callbacks being called multiple times (ZOOKEEPER-502), notify() on the control object is called too many times, resulting in the unit test not waiting for certain callbacks.

Thus the test never waits for the asyncOpenLedger() to finish, and hence still succeeds. I believe asyncOpenLedger() has never worked right.

47765

No Perforce job exists for this issue.

1

33034

08/Aug/09 8:49 PM

10 years, 9 weeks, 2 days ago

0|i060mv:

ZooKeeper

ZOOKEEPER-504

ClassCastException in LedgerManagementProcessor

Bug

Closed

Major

Fixed

Utkarsh Srivastava

07/Aug/09 13:35

26/Mar/10 13:24

26/Jan/10 18:17

3.2.0

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-503, ZOOKEEPER-507

java.lang.ClassCastException: org.apache.bookkeeper.client.LedgerManagementProcessor$OpenLedgerOp cannot be cast to org.apache.bookkeeper.client.LedgerManagementProcessor$CloseLedgerOp
at org.apache.bookkeeper.client.LedgerManagementProcessor.processResult(LedgerManagementProcessor.java:1083)

This seems to be happening because its a nested switch case statement. And the OPEN: case, doesn't ever call a break. It only calls a break from the inner switch-case and hence falls through into the CLOSE: case.

47766

No Perforce job exists for this issue.

3

33035

07/Aug/09 10:23 PM

10 years, 9 weeks, 2 days ago

0|i060n3:

ZooKeeper

ZOOKEEPER-503

race condition in asynchronous create

Bug

Closed

Major

Fixed

Benjamin Reed

06/Aug/09 21:06

01/May/13 22:29

26/Jan/10 18:18

3.3.0

contrib-bookkeeper

0

3

ZOOKEEPER-502, ZOOKEEPER-504, ZOOKEEPER-461, ZOOKEEPER-507

there is a race condition between the zookeeper completion thread and the bookeeper processing queue during create. if the zookeeper completion thread falls behind due to scheduling, the action counter of the create operation may go backwards.

47767

No Perforce job exists for this issue.

2

33036

07/Aug/09 10:36 AM

10 years, 9 weeks, 2 days ago

0|i060nb:

ZooKeeper

ZOOKEEPER-502

bookkeeper create calls completion too many times

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Benjamin Reed

06/Aug/09 20:57

26/Mar/10 13:24

26/Jan/10 18:19

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-503, ZOOKEEPER-507

when calling the asynchronous version of create, the completion routine is called more than once.

47768

No Perforce job exists for this issue.

1

33037

08/Aug/09 2:19 AM

10 years, 9 weeks, 2 days ago

0|i060nj:

ZooKeeper

ZOOKEEPER-501

CnxManagerTest failed on hudson

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

06/Aug/09 18:11

05/Sep/09 18:36

07/Aug/09 17:03

3.2.1, 3.3.0

0

2

It timed out according to the console output:

http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/406/testReport/org.apache.zookeeper.test/CnxManagerTest/testCnxManager/

47769

No Perforce job exists for this issue.

2

33038

06/Aug/09 6:45 PM

10 years, 33 weeks, 5 days ago

Reviewed

0|i060nr:

ZooKeeper

ZOOKEEPER-500

Async methods shouldnt throw exceptions

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

Utkarsh Srivastava

05/Aug/09 19:01

23/Nov/11 14:22

06/Jan/11 11:08

3.4.0

contrib-bookkeeper

0

Async methods like asyncLedgerCreate and Open shouldnt be throwing InterruptedException and BKExceptions.

The present method signatures lead to messy application code since one is forced to have error handling code in 2 places: inside the callback to handler a non-OK return code, and outside for handling the exceptions thrown by the call.

There should be only one way to indicate error conditions, and that should be through a non-ok return code to the callback.

47770

No Perforce job exists for this issue.

1

33454

18/Dec/10 10:00 AM

9 years, 11 weeks, 6 days ago

Reviewed

0|i06387:

ZooKeeper

ZOOKEEPER-499

electionAlg should default to FLE (3) - regression

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

05/Aug/09 16:32

05/Sep/09 18:36

10/Aug/09 18:25

3.2.0

3.2.1, 3.3.0

server, tests

0

2

there's a regression in 3.2 - electionAlg is no longer defaulting to 3 (incorrectly defaults to 0)

also - need to have tests to validate this

47771

No Perforce job exists for this issue.

4

33039

06/Aug/09 10:50 PM

10 years, 33 weeks, 2 days ago

workaround in 3.2.0 (this only effects 3.2.0)

set electionAlg=3 in server config files.

Reviewed

0|i060nz:

ZooKeeper

ZOOKEEPER-498

Unending Leader Elections : WAN configuration

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

Todd Greenwood-Geer

04/Aug/09 17:18

05/Sep/09 18:36

11/Aug/09 17:27

3.2.0

3.2.1, 3.3.0

leaderElection

0

2

Each machine:

CentOS 5.2 64-bit
2GB ram
java version "1.6.0_13"
Java(TM) SE Runtime Environment (build 1.6.0_13-b03)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02, mixed

Network Topology:
DC : central data center
POD(N): remote data center

Zookeeper Topology:
Leaders may be elected only in DC (weight = 1)
Only followers are elected in PODS (weight = 0)

In a WAN configuration, ZooKeeper is endlessly electing, terminating, and re-electing a ZooKeeper leader. The WAN configuration involves two groups, a central DC group of ZK servers that have a voting weight = 1, and a group of servers in remote pods with a voting weight of 0.

What we expect to see is leaders elected only in the DC, and the pods to contain only followers. What we are seeing is a continuous cycling of leaders. We have seen this consistently with 3.2.0, 3.2.0 + recommended patches (473, 479, 481, 491), and now release 3.2.1.

47772

No Perforce job exists for this issue.

9

33040

05/Aug/09 12:59 PM

10 years, 33 weeks, 1 day ago

Reviewed

0|i060o7:

ZooKeeper

ZOOKEEPER-497

api and forrest docs should mention if classes are thread safe

Bug

Open

Minor

Unresolved

Patrick D. Hunt

04/Aug/09 16:51

14/Dec/19 06:07

3.2.0

3.7.0

documentation

0

the api (c/java clients) and the forrest docs should talk about thread safety - in particular we don't
mention that ZooKeeper class is thread safe (etc...) Docs should be updated.

70753

No Perforce job exists for this issue.

0

33041

10 years, 2 weeks, 6 days ago

0|i060of:

ZooKeeper

ZOOKEEPER-496

zookeeper-tree utility for export, import and incremental updates

New Feature

Closed

Major

Fixed

Anirban Roy

03/Aug/09 14:51

26/Mar/10 13:24

12/Jan/10 17:02

3.3.0

contrib

0

6

604800

0%

RHEL 4.6, libxml2

==========================================
zktreeutil - Zookeeper Tree Data Utility
Author: Anirban Roy
Organization: Yahoo Inc.
==========================================

zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
for any cluster managed by Zookeeper. Here are the basic functionalities -

EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
capturing a current snapshot of the data for backup/analysis. For a subtree
export, one need to specify the path to the ZK-subtree with proper option.

IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
the new ZK cluster with static configuration. The import can be non-intrusive by
making only the additions in the existing data. The import of subtree is also
possible by optionally providing the path to the ZK-subtree.

DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
diff command.

UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
lly after running the diff.

DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
server or XML file. Like export, ZK-subtree can be dumped with optionaly
providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

The exported ZK data into XML file can be shortened by only keeping the static ZK
nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

0%

604800

47773

No Perforce job exists for this issue.

2

33455

11/Aug/09 1:52 AM

10 years, 10 weeks, 6 days ago

==========================================
zktreeutil - Zookeeper Tree Data Utility
Author: Anirban Roy
Organization: Yahoo Inc.
==========================================

zktreeutil program is intended to manage and manipulate zk-tree data quickly, effi-
ciently and with ease. The utility operates on free-form ZK-tree and hence can be used
for any cluster managed by Zookeeper. Here are the basic functionalities -

EXPORT: The whole/partial ZK-tree is exported into a XML file. This helps in
capturing a current snapshot of the data for backup/analysis. For a subtree
export, one need to specify the path to the ZK-subtree with proper option.

IMPORT: The ZK-tree can be imported from XML into ZK cluster. This helps in priming
the new ZK cluster with static configuration. The import can be non-intrusive by
making only the additions in the existing data. The import of subtree is also
possible by optionally providing the path to the ZK-subtree.

DIFF: Creates a diff between live ZK data vs data saved in XML file. Diff can ignore
some ZK-tree branches (possibly dynamic data) on reading the optional ignore flag
from XML file. Diffing on a ZK-subtree achieved by providing path to ZK-subtree with
diff command.

UPDATE: Make the incremental changes into the live ZK-tree from saved XML, essentia-
lly after running the diff.

DUMP: Dumps the ZK-tree on the standard output device reading either from live ZK
server or XML file. Like export, ZK-subtree can be dumped with optionaly
providing the path to the ZK-subtree, and till a certain depth of the (sub)tree.

The exported ZK data into XML file can be shortened by only keeping the static ZK
nodes which are required to prime a cluster. The dynamic zk nodes (created on-the-
fly) can be ignored by setting a 'ignore' attribute at the root node of the dynamic
subtree (see tests/zk_sample.xml), possibly deleting all inner ZK nodes under that.
Once ignored, the whole subtree is ignored during DIFF, UPDATE and WRITE.

Pre-requisites
--------------
1. Linux system with 2.6.X kernel.
2. Zookeeper C client library (locally built at ../../c/.libs) >= 3.X.X
3. Development build libraries (rpm packages):
a. boost-devel >= 1.32.0
b. libxml2-devel >= 2.7.3
c. log4cxx0100-devel >= 0.10.0

Build instructions
------------------
1. cd into this directory
2. autoreconf -if
3. ./configure
4. make
5. 'zktreeutil' binary created under src directory

Limitations
-----------
Current version works with text data only, binary data will be supported in future
versions.

Testing and usage of zktreeutil
--------------------------------
1. Run Zookeeper server locally on port 2181
2. export LD_LIBRARY_PATH=../../c/.libs/:/usr/local/lib/
3. ./src/zktreeutil --help # show help
4. ./src/zktreeutil --zookeeper=localhost:2181 --import --xmlfile=tests/zk_sample.xml 2>/dev/null # import sample ZK tree
5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --path=/myapp/version-1.0 2>/dev/null # dump Zk subtree
5. ./src/zktreeutil --zookeeper=localhost:2181 --dump --depth=3 2>/dev/null # dump Zk tree till certain depth
6. ./src/zktreeutil --xmlfile=zk_sample.xml -D 2>/dev/null # dump the xml data
7. Change zk_sample.xml with adding/deleting/chaging some nodes
8. ./src/zktreeutil -z localhost:2181 -F -x zk_sample.xml -p /myapp/version-1.0/configuration 2>/dev/null # take a diff of changes
9. ./src/zktreeutil -z localhost:2181 -E 2>/dev/null > zk_sample2.xml # export the mofied ZK tree
10. ./src/zktreeutil -z localhost:2181 -U -x zk_sample.xml -p /myapp/version-1.0/distributions 2>/dev/null # update with incr. changes
11. ./src/zktreeutil --zookeeper=localhost:2181 --import --force --xmlfile=zk_sample2.xml 2>/dev/null # re-prime the ZK tree

Reviewed

zookeeper tree trree-data utility zktree

0|i0638f:

ZooKeeper

ZOOKEEPER-495

c client logs an invalid error when zookeeper_init is called with chroot

Bug

Closed

Minor

Fixed

Patrick D. Hunt

Michi Mutsuzaki

03/Aug/09 14:20

07/May/10 13:40

25/Jan/10 22:11

3.2.0

3.3.0

c client

0

2

The C client logs this error message when zookeeper_init is called with chroot.

2009-08-03 18:14:29,130:6624(0x5e66e950):ZOO_ERROR@sub_string@730: server path does not include chroot path /chroot

I'll attach a simple program to reproduce this.

Thanks!
--Michi

logging

47774

No Perforce job exists for this issue.

3

33042

22/Jan/10 7:18 PM

10 years, 9 weeks, 1 day ago

Reviewed

0|i060on:

ZooKeeper

ZOOKEEPER-494

zookeeper should install include headers in /usr/local/include/zookeeper

Improvement

Closed

Trivial

Duplicate

Isabel Drost-Fromm

Michi Mutsuzaki

02/Aug/09 15:07

23/Nov/11 14:22

04/May/11 01:49

3.2.0

3.4.0

c client

0

3

ZOOKEEPER-1033

Hello,

Currently all the c client header files get installed under /usr/local/include/c-client-src . Ideally they should get installed in /usr/local/include/zookeeper .

214122

No Perforce job exists for this issue.

1

33456

28/Aug/09 3:09 AM

8 years, 47 weeks, 1 day ago

Changed install path of c-client header files to /usr/local/include/zookeeper

0|i0638n:

ZooKeeper

ZOOKEEPER-493

patch for command line setquota

Bug

Closed

Minor

Fixed

steve bendiola

01/Aug/09 05:40

05/Sep/09 18:36

04/Aug/09 17:39

3.2.0

3.2.1, 3.3.0

java client

0

2

the command line "setquota" tries to use argument 3 as both a path and a value

47775

No Perforce job exists for this issue.

2

33043

04/Aug/09 5:36 PM

10 years, 34 weeks, 1 day ago

Reviewed

0|i060ov:

ZooKeeper

ZOOKEEPER-492

the tests should have their own log4j.properties

Bug

Resolved

Minor

Invalid

Unassigned

Patrick D. Hunt

31/Jul/09 00:24

23/Apr/14 16:40

3.1.1, 3.2.0

3.5.0

tests

0

2

the tests currently rely (QuorumPeerMainTest in particular) on conf/log4j.properties. if the user changes this file
and then runs the tests the tests may fail. the tests should have their own log4j.properties maintained w/in the test
directory itself, separate from conf/log4j.properties

70795

No Perforce job exists for this issue.

0

33044

16/Aug/10 2:26 AM

5 years, 48 weeks, 1 day ago

0|i060p3:

ZooKeeper

ZOOKEEPER-491

Prevent zero-weight servers from being elected

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

29/Jul/09 14:51

01/May/13 22:29

04/Aug/09 14:17

3.2.0

3.2.1, 3.3.0

leaderElection

0

2

ZOOKEEPER-479, ZOOKEEPER-481

This is a fix to prevent zero-weight servers from being elected leaders. This will allow in wide-area scenarios to restrict the set of servers that can lead the ensemble.

47776

No Perforce job exists for this issue.

2

33045

04/Aug/09 2:17 PM

10 years, 34 weeks, 1 day ago

Reviewed

0|i060pb:

ZooKeeper

ZOOKEEPER-490

the java docs for session creation are misleading/incomplete

Bug

Closed

Major

Fixed

Patrick D. Hunt

29/Jul/09 14:27

05/Sep/09 18:36

07/Aug/09 14:17

3.1.1, 3.2.0

3.2.1, 3.3.0

0

2

the javadoc for ZooKeeper constructor says:

* The client object will pick an arbitrary server and try to connect to it.
* If failed, it will try the next one in the list, until a connection is
* established, or all the servers have been tried.

the "or all server tried" phrase is misleading, it should indicate that we retry until success, con closed, or session expired.

we also need ot mention that connection is async, that constructor returns immed and you need to look for connection event in watcher

47777

No Perforce job exists for this issue.

1

33046

06/Aug/09 3:58 PM

10 years, 33 weeks, 5 days ago

Reviewed

0|i060pj:

ZooKeeper

ZOOKEEPER-489

findbugs on hudson.zones.a.o has jumped way up (2k+)

Bug

Closed

Minor

Cannot Reproduce

Giridharan Kesavan

Patrick D. Hunt

28/Jul/09 02:40

26/Mar/10 13:24

29/Sep/09 05:59

3.3.0

build

0

1

for some reason hudson.zones violations report has jumped up to 2000+ findbugs issues in just the last couple of weeks. It's
jumped up from 0 violations so something is weird...

http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/

Giri can you take a look?

214121

No Perforce job exists for this issue.

0

33047

02/Sep/09 2:24 AM

10 years, 7 weeks, 2 days ago

0|i060pr:

ZooKeeper

ZOOKEEPER-488

to fix src/c/tests/zkServer.sh script to add clover in classpath

Improvement

Closed

Major

Fixed

Giridharan Kesavan

27/Jul/09 02:19

26/Mar/10 13:24

27/Jul/09 04:31

3.3.0

scripts

0

1

47778

No Perforce job exists for this issue.

1

33457

27/Jul/09 3:04 AM

10 years, 35 weeks, 3 days ago

0|i0638v:

ZooKeeper

ZOOKEEPER-487

setdata on root (/) crashes the servers.

Bug

Closed

Blocker

Fixed

Mahadev Konar

24/Jul/09 16:38

05/Sep/09 18:36

31/Jul/09 15:08

3.1.1, 3.2.0

3.2.1, 3.3.0

server

0

2

setdata on root "/" crashes the servers with the followimg exception. Unfortunately we never had a setdata test on root. The following is the exception. This happens with 3.1.1 as well. We might want to consider releasing 3.1.2 just for this jira.

{code}
java.lang.IllegalArgumentException: Invalid path /
at org.apache.zookeeper.common.PathTrie.findMaxPrefix(PathTrie.java:255)
at org.apache.zookeeper.server.DataTree.setData(DataTree.java:543)
at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:701)
at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:94)
at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:127)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:75)
{code}

47779

No Perforce job exists for this issue.

1

33048

24/Jul/09 8:56 PM

10 years, 34 weeks, 5 days ago

Reviewed

0|i060pz:

ZooKeeper

ZOOKEEPER-486

Improve bookie performance for large number of ledgers

Improvement

Closed

Major

Fixed

Benjamin Reed

Flavio Paiva Junqueira

23/Jul/09 21:03

26/Mar/10 13:24

13/Mar/10 09:24

3.3.0

contrib-bookkeeper

0

1

If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.

47780

No Perforce job exists for this issue.

5

33458

22/Oct/09 5:26 PM

10 years, 2 weeks, 5 days ago

0|i06393:

ZooKeeper

ZOOKEEPER-485

need ops documentation that details supervision of ZK server processes

Bug

Closed

Major

Fixed

Patrick D. Hunt

23/Jul/09 17:47

26/Mar/10 13:24

24/Feb/10 15:04

3.3.0

documentation, server

0

3

ZOOKEEPER-483

We need ops documentation detailing what to do if the ZK server VM fails - by fail I mean the jvm process
exits/dies/crashes/etc...

In general a supervisor process should be used to start/stop/restart/etc... the ZK server vm.

Something like daemontools http://cr.yp.to/daemontools.html could be used, or more simply a wrapper script
should monitor the status of the pid and restart if the jvm fails. It's up to the operator, if this is not done
automatically then it will have to be done manually, by operator restarting the ZK server jvm

The inherent behavior of ZK wrt to failures - ie that it automatically recovers as long as quorum is maintained -
fits into this nicely.

47781

No Perforce job exists for this issue.

1

33049

24/Jul/09 1:44 AM

10 years, 5 weeks ago

Reviewed

0|i060q7:

ZooKeeper

ZOOKEEPER-484

Clients get SESSION MOVED exception when switching from follower to a leader.

Bug

Closed

Blocker

Fixed

Mahadev Konar

23/Jul/09 17:09

05/Sep/09 18:36

06/Aug/09 20:39

3.2.0

3.2.1, 3.3.0

server

0

3

When a client is connected to follower and get disconnected and connects to a leader it gets SESSION MOVED excpetion. This is beacuse of a bug in the new feature of ZOOKEEPER-417 that we added in 3.2. All the releases before 3.2 DO NOT have this problem. The fix is to make sure the ownership of a connection gets changed when a session moves from follower to the leader. The workaround to it in 3.2.0 would be to swithc off connection from clients to the leader. take a look at *leaderServers* java property in http://hadoop.apache.org/zookeeper/docs/r3.2.0/zookeeperAdmin.html.

47782

No Perforce job exists for this issue.

3

33050

23/Jul/09 8:59 PM

10 years, 33 weeks, 6 days ago

Reviewed

0|i060qf:

ZooKeeper

ZOOKEEPER-483

ZK fataled on me, and ugly

Bug

Closed

Major

Fixed

Benjamin Reed

ryan rawson

23/Jul/09 16:40

05/Sep/09 18:36

25/Aug/09 01:58

3.1.1

3.2.1, 3.3.0

0

4

ZOOKEEPER-508, ZOOKEEPER-509, ZOOKEEPER-485

here are the part of the log whereby my zookeeper instance crashed, taking 3 out of 5 down, and thus ruining the quorum for all clients:

2009-07-23 12:29:06,769 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x52276d1d5161350 due to java.io.IOException: Read error
2009-07-23 12:29:00,756 WARN org.apache.zookeeper.server.quorum.Follower: Exception when following the leader
java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:65)
at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
at org.apache.zookeeper.server.quorum.Follower.readPacket(Follower.java:114)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:243)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:494)
2009-07-23 12:29:06,770 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5161350 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.168:39489]
2009-07-23 12:29:06,770 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb0578 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46797]
2009-07-23 12:29:06,771 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa013e NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.153:33998]
2009-07-23 12:29:06,771 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x52276d1d5160593 due to java.io.IOException: Read error
2009-07-23 12:29:06,808 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e02bb NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.158:53758]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa13e4 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.154:58681]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691382 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59967]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb1354 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.163:49957]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa13cd NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.150:34212]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691383 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46813]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb0350 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59956]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e139b NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.156:55138]
2009-07-23 12:29:06,809 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e1398 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.167:41257]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5161355 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.153:34032]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d516011c NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56314]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e056b NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56322]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d516011f NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.157:49618]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e11ea NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.10.20.42:55483]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e02ba NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.157:49632]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb1355 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.169:58824]
2009-07-23 12:29:06,810 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691378 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.161:40973]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691380 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59944]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e0311 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.160:56167]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e690374 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.169:58815]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e139f NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.151:51396]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e139c NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56315]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137b NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59859]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5160594 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.151:51370]
2009-07-23 12:29:06,811 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137a NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46682]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5160347 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.165:35722]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137f NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46754]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5160121 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56307]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb0126 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.154:58688]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa05fc NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.152:45067]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e0316 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.169:58800]
2009-07-23 12:29:06,812 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137e NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46737]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137d NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.159:46733]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa13df NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.156:55137]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb134e NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.166:40443]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691381 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.161:41086]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5161356 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.165:35719]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb1349 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.
20.20.158:53770]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x12276d15dfb0352 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.165:35718]
2009-07-23 12:29:06,813 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691379 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59823]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d516000e NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.150:34216]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x32276d15d2e1397 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.169:58829]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e69137c NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59862]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa0140 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56271]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x42276d1d3fa13e1 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.157:49608]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x22276d15e691377 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.162:59789]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x52276d1d5160593 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.165:35703]
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.FinalRequestProcessor: shutdown of request processor complete
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.quorum.FollowerRequestProcessor: FollowerRequestProcessor exited loop!
2009-07-23 12:29:06,814 INFO org.apache.zookeeper.server.quorum.CommitProcessor: CommitProcessor exited loop!
2009-07-23 12:29:06,815 INFO org.apache.zookeeper.server.quorum.Follower: shutdown called
java.lang.Exception: shutdown Follower
at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:427)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:498)
2009-07-23 12:29:06,815 WARN org.apache.zookeeper.server.NIOServerCnxn: Ignoring exception
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
at org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:201)
2009-07-23 12:29:06,815 INFO org.apache.zookeeper.server.quorum.QuorumPeer: LOOKING
2009-07-23 12:29:06,817 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
2009-07-23 12:29:06,817 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x0 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.156:55206]
2009-07-23 12:29:06,818 WARN org.apache.zookeeper.server.NIOServerCnxn: Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
2009-07-23 12:29:06,818 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x0 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.155:56331]
[elided lots of the same]
2009-07-23 12:29:33,008 INFO org.apache.zookeeper.server.NIOServerCnxn: closing session:0x0 NIOServerCnxn: java.nio.channels.SocketChannel[connected local=/10.20.20.151:2181 remote=/10.20.20.152:5945
8]
2009-07-23 12:29:33,011 FATAL org.apache.zookeeper.server.SyncRequestProcessor: Severe unrecoverable error, exiting
java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at org.apache.zookeeper.server.quorum.Follower.writePacket(Follower.java:100)
at org.apache.zookeeper.server.quorum.SendAckRequestProcessor.flush(SendAckRequestProcessor.java:52)
at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:131)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:76)

The good news is when I restarted the downed zookeepers, everything returned to normal.

47783

No Perforce job exists for this issue.

7

33051

23/Jul/09 5:51 PM

10 years, 31 weeks, 2 days ago

ZOOKEEPER-508 includes the fix.

Reviewed

0|i060qn:

ZooKeeper

ZOOKEEPER-482

ignore sigpipe in testRetry to avoid silent immediate failure

Bug

Closed

Minor

Fixed

Chris Darroch

22/Jul/09 18:44

05/Sep/09 18:36

30/Jul/09 18:34

3.2.0

3.2.1, 3.3.0

c client, tests

0

2

The testRetry test silently exits for me periodically, especially, it seems, on newer hardware. It also spits out from log messages clutter the test output.

The silent exits turn out to be because SIGPIPE is sometimes delivered during the sleep(1) in createClient(), the second time createClient() is called. Since SIGPIPE is not being ignored and there is no signal handler, the process exists immediately. This leaves the test suite in a broken state, with the test ZooKeeper process still running because "zkServer.sh stop" is not run by tearDown(). You have to manually kill the ZK server and retry the tests; sometimes they succeed and sometimes they don't.

I described SIGPIPE handling a little in ZOOKEEPER-320. The appropriate thing, I think, is for the client application to ignore or handle SIGPIPE. In this case, that falls to the test processes. The attached patch fixes the issue for me with testRetry.

The patch uses sigaction() to ignore SIGPIPE in TestClientRetry.cc and, for good measure (although I never saw it actually fail for me), TestClient.cc, since that file also uses sleep() extensively.

I also removed a couple of unused functions and a macro definition from TestClientRetry.cc, just to simply matters, and turned off log output, which makes the testRetry output much, much cleaner (otherwise you get a lot of log output spamming into the nice clean cppunit output :-).

47784

No Perforce job exists for this issue.

1

33052

22/Jul/09 6:49 PM

10 years, 34 weeks, 6 days ago

Ignore SIGPIPE signal to C test suite does not fail suddenly.

Reviewed

0|i060qv:

ZooKeeper

ZOOKEEPER-481

Add lastMessageSent to QuorumCnxManager

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

20/Jul/09 09:43

01/May/13 22:29

03/Aug/09 17:32

3.1.1, 3.2.0

3.2.1, 3.3.0

leaderElection

0

2

ZOOKEEPER-475, ZOOKEEPER-479, ZOOKEEPER-491

Currently we rely on TCP for reliable delivery of FLE messages. However, as we concurrently drop and create new connections, it is possible that a message is sent but never received. With this patch, cnx manager keeps a list of last messages sent, and resends the last one sent. Receiving multiples copies is harmless.

47785

No Perforce job exists for this issue.

7

33053

23/Jul/09 7:25 AM

10 years, 34 weeks, 2 days ago

Reviewed

0|i060r3:

ZooKeeper

ZOOKEEPER-480

FLE should perform leader check when node is not leading and add vote of follower

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

20/Jul/09 09:33

01/May/13 22:29

04/Aug/09 14:07

3.2.0

3.2.1, 3.3.0

0

2

ZOOKEEPER-475

As a server may join leader election while others have already elected a leader, it is necessary that a server handles some special cases of leader election when notifications are from servers that are either LEADING or FOLLOWING. In such special cases, we check if we have received a message from the leader to declare a leader elected. This check does not consider the case that the process performing the check might be a recently elected leader, and consequently the check fails.

This patch also adds a new case, which corresponds to adding a vote to recvset when the notification is from a process LEADING or FOLLOWING. This fixes the case raised in ZOOKEEPER-475.

47786

No Perforce job exists for this issue.

7

33054

30/Jul/09 7:27 AM

10 years, 34 weeks, 1 day ago

Reviewed

0|i060rb:

ZooKeeper

ZOOKEEPER-479

QuorumHierarchical does not count groups correctly

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

20/Jul/09 09:00

01/May/13 22:29

03/Aug/09 18:30

3.2.1, 3.3.0

quorum

0

2

ZOOKEEPER-475, ZOOKEEPER-481, ZOOKEEPER-491

QuorumHierarchical::containsQuorum should not verify if all groups represented in the input set have more than half of the total weight. Instead, it should check only for an overall majority of groups.

47787

No Perforce job exists for this issue.

4

33055

30/Jul/09 4:31 AM

10 years, 34 weeks, 2 days ago

Reviewed

0|i060rj:

ZooKeeper

ZOOKEEPER-478

support custom hostnames for client and quorum connections

Improvement

Open

Minor

Unresolved

Unassigned

Chris Darroch

17/Jul/09 15:36

3.2.0

quorum, server

0

1

Our system administrators would love it if we could configure ZooKeeper to listen for client and quorum connections on a hostname which isn't bound to the localhost.

Maybe there's some neat way to do this I'm not aware of already, of course, but it looks to me like we would need to change the two ss.socket().bind(new InetSocketAddress(port)); calls, one in NIOServerCnxn and one in QuorumCnxManager to so that they instead used InetSocketAddress(host, port). Obviously that implies some optional definition of a hostname in the config file as well and possibly on the command-line.

Does that seem like the right approach?

214120

No Perforce job exists for this issue.

0

68463

10 years, 36 weeks, 6 days ago

0|i0c37b:

ZooKeeper

ZOOKEEPER-477

zkCleanup.sh is flaky

Bug

Closed

Major

Fixed

Fernando Padilla

17/Jul/09 15:05

05/Sep/09 18:36

10/Aug/09 20:30

3.2.0

3.2.1, 3.3.0

scripts

0

2

the zkCleanup.sh script is buggy in two ways:

1) it doesn't actually pass through the snapshot count, so it doesn't work
2) it assumes that there is only dataDir, it doesn't support dataLogDir

And it can use cleanup, so that it doesn't blindly call eval from the config file..

47788

No Perforce job exists for this issue.

2

33056

17/Jul/09 4:14 PM

10 years, 33 weeks, 2 days ago

Reviewed

0|i060rr:

ZooKeeper

ZOOKEEPER-476

upgrade junit library from 4.4 to 4.6

Improvement

Closed

Major

Invalid

Patrick D. Hunt

16/Jul/09 18:57

26/Mar/10 13:24

18/Sep/09 19:14

3.3.0

tests

0

1

upgrade from junit 4.4 to 4.6

214119

No Perforce job exists for this issue.

2

33459

28/Jul/09 4:09 AM

10 years, 27 weeks, 6 days ago

Reviewed

0|i0639b:

ZooKeeper

ZOOKEEPER-475

FLENewEpochTest failed on nightly builds.

Bug

Closed

Blocker

Fixed

Flavio Paiva Junqueira

Mahadev Konar

16/Jul/09 17:39

01/May/13 22:29

04/Aug/09 14:18

3.2.0

3.2.1, 3.3.0

quorum

0

2

ZOOKEEPER-479, ZOOKEEPER-480, ZOOKEEPER-481

THe flenewepochtest failed on one of the nightly builds -
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/377.

47789

No Perforce job exists for this issue.

2

33057

17/Jul/09 6:30 AM

10 years, 34 weeks, 2 days ago

0|i060rz:

ZooKeeper

ZOOKEEPER-474

add compile, test, and improved package targets to zkperl build.xml

Improvement

Open

Major

Unresolved

Chris Darroch

15/Jul/09 14:06

14/Dec/19 06:07

3.2.0

3.7.0

contrib

0

ZOOKEEPER-466, ZOOKEEPER-471

This patch adds compile and test targets to the zkperl build.xml, and tweaks the package target a little to use the manifest file.

For me, "ant compile", "ant test", and "ant clean" all work (from scratch, in each case) when using Ant in the local src/contrib/zkperl directory.

Further, "ant package" in the top-level directory seems to continue to build zkperl along with everything else, and leaves out the build.xml and t/zkServer.sh files, which is appropriate.

From what I can see, the top-level build.xml doesn't actually invoke the test-contrib target, so I'm not sure if there's a way to integrate the zkperl tests into the main hudson automated test process, but that would be ideal, if at all possible. I feel like I've seen comments to the effect that the zkpython tests are run automatically, but I'm not sure if that's actually true or not.

70756

No Perforce job exists for this issue.

2

68464

04/Aug/09 12:23 AM

10 years, 33 weeks, 3 days ago

Add compile, test, and improved package targets to zkperl build.xml.

0|i0c37j:

ZooKeeper

ZOOKEEPER-473

cleanup junit tests to eliminate false positives due to "socket reuse" and failure to close client

Improvement

Closed

Major

Fixed

Patrick D. Hunt

15/Jul/09 12:46

26/Mar/10 13:24

16/Jul/09 18:10

3.2.0

3.3.0

tests

0

1

ZOOKEEPER-460

The junit tests will occasionally fail on hudson due to a few problems:

1) socket in use - sometimes the code/testcode will fail to close a socket, either that or tcp wait
kicks in and the kernel holds onto the socket, either way subsequent tests that use this same
port will fail

2) sometimes the tests fail to close clients, this results in clients attempting to re-connect to
the server started as part of a subsequent test (the timeout is very long). this causes a few
problems, typically extra load on the server or more likely unusual changes to the server that the
next test(s) don't expect. also causes problems when trying to debug the test as the old clients
spew messages into the log.

we see this on hudson, occasionally failures. We also saw this when working on ZOOKEEPER-397, the testng
port. testng does not support "fork mode" for running the tests. as a result all the tests (all) run in a single
vm instance, start to finish. so this problem is exacerbated significantly in that case. this patch only
fixes the problem in junit btw, it does not move to testng

47790

No Perforce job exists for this issue.

2

33460

15/Jul/09 6:37 PM

10 years, 35 weeks, 3 days ago

Reviewed

0|i0639j:

ZooKeeper

ZOOKEEPER-472

Making DataNode not instantiate a HashMap when the node is ephmeral

Improvement

Closed

Minor

Fixed

Erik Holstad

14/Jul/09 19:44

26/Mar/10 13:24

12/Nov/09 17:48

3.1.1, 3.2.0

3.3.0

server

1

3

Looking at the code, there is an overhead of a HashSet object for that nodes children, even though the node might be an ephmeral node and cannot have children.

47791

No Perforce job exists for this issue.

6

33461

14/Jul/09 8:10 PM

10 years, 19 weeks, 5 days ago

Reviewed

0|i0639r:

ZooKeeper

ZOOKEEPER-471

update zkperl for 3.2.x branch

Improvement

Closed

Minor

Fixed

Chris Darroch

14/Jul/09 15:05

01/May/13 22:29

16/Jul/09 21:14

3.2.0

3.2.1, 3.3.0

contrib

0

1

ZOOKEEPER-474

Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme.

47792

No Perforce job exists for this issue.

1

33462

16/Jul/09 8:56 PM

10 years, 35 weeks, 3 days ago

Update zkperl contrib module for 3.2.x branch.

Reviewed

0|i0639z:

ZooKeeper

ZOOKEEPER-470

include unistd.h for sleep() in c tests

Bug

Closed

Minor

Fixed

Chris Darroch

14/Jul/09 14:25

05/Sep/09 18:36

20/Jul/09 18:55

3.2.0

3.2.1, 3.3.0

c client

0

2

Include unistd.h for sleep() calls in C tests to ensure successful compilation on some platforms.

47793

No Perforce job exists for this issue.

1

33058

14/Jul/09 8:04 PM

10 years, 35 weeks, 3 days ago

Include unistd.h for sleep() calls in C tests.

Reviewed

0|i060s7:

ZooKeeper

ZOOKEEPER-469

make sure CPPUNIT_CFLAGS isn't overwritten

Bug

Closed

Major

Fixed

Chris Darroch

14/Jul/09 14:10

05/Sep/09 18:36

16/Jul/09 19:30

3.2.0

3.2.1, 3.3.0

c client

0

2

When compiling with --with-cppunit-prefix, CPPUNIT_CFLAGS is set by the AM_PATH_CPPUNIT macro. In configure.ac, it is then reset in order to set the -DZKSERVER_CMD command line argument. Instead, that argument should be added to CPPUNIT_CFLAGS so that things like a custom -I include location set by AM_PATH_CPPUNIT are not lost. Otherwise, a custom cppunit installation is not properly supported, despite the --with-cppunit-prefix option.

47794

No Perforce job exists for this issue.

1

33059

15/Jul/09 7:09 PM

10 years, 35 weeks, 3 days ago

Ensure custom CPPUNIT_CFLAGS is not overwritten.

Reviewed

0|i060sf:

ZooKeeper

ZOOKEEPER-468

avoid compile warning in send_auth_info()

Bug

Closed

Minor

Fixed

Chris Darroch

14/Jul/09 13:44

05/Sep/09 18:36

16/Jul/09 13:42

3.2.0

3.2.1, 3.3.0

c client

0

2

Older compilers may complain that rc may be used without initialization in send_auth_info(), if -Wall is specified. The fix is a simple initialization.

47795

No Perforce job exists for this issue.

1

33060

16/Jul/09 10:59 AM

10 years, 35 weeks, 3 days ago

Fix minor compile warning in send_auth_info().

0|i060sn:

ZooKeeper

ZOOKEEPER-467

Change log level in BookieHandle

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

14/Jul/09 03:47

05/Sep/09 18:36

30/Jul/09 17:54

3.2.1, 3.3.0

contrib-bookkeeper

0

2

I have accidentally left a log.warn in BookieHandle. This is a pretty simple patch, so I would appreciate if we could review it quickly.

47796

No Perforce job exists for this issue.

1

33061

28/Jul/09 1:23 AM

10 years, 34 weeks, 6 days ago

Reviewed

0|i060sv:

ZooKeeper

ZOOKEEPER-466

crash on zookeeper_close() when using auth with empty cert

Bug

Closed

Major

Fixed

Chris Darroch

13/Jul/09 23:25

01/May/13 22:29

03/Aug/09 21:59

3.2.0

3.2.1, 3.3.0

c client

0

2

ZOOKEEPER-474

The free_auth_info() function calls deallocate_Buffer(&auth->auth) on every element in the auth list; that function frees any memory pointed to by auth->auth.buff if that field is non-NULL.

In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo->auth when auth.buff is NULL. The result is uninitialized data in auth->auth.buff in free_auth_info(), and potential crashes.

The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe.

47797

No Perforce job exists for this issue.

1

33062

03/Aug/09 9:59 PM

10 years, 34 weeks, 2 days ago

Fixes crash in zookeeper_close() when zoo_add_auth() used with empty cert string.

Reviewed

0|i060t3:

ZooKeeper

ZOOKEEPER-465

Ledger size in bytes

New Feature

Closed

Major

Fixed

Flavio Paiva Junqueira

13/Jul/09 16:24

23/Nov/11 14:22

17/Feb/11 16:53

3.3.3, 3.4.0

contrib-bookkeeper

0

4

It is currently easy to know how many entries a ledger has, but there is no easy way to know the total number of bytes in a ledger. The idea of this jira is to add a method that gives the number of bytes in a closed ledger. My current idea is to simply have the writer counting the number of bytes written and store it to ZooKeeper.

47798

No Perforce job exists for this issue.

5

33463

17/Dec/10 1:50 PM

9 years, 6 weeks ago

Reviewed

0|i063a7:

ZooKeeper

ZOOKEEPER-464

Need procedure to garbage collect ledgers

New Feature

Closed

Major

Fixed

Erwin Tam

Flavio Paiva Junqueira

13/Jul/09 16:20

23/Nov/11 14:22

13/May/10 16:24

3.4.0

contrib-bookkeeper

0

An application using BookKeeper is likely to use a large number of ledgers over time. Such an application might not need all ledgers created over time and might want to delete some of these ledgers to free up some space on bookies. The idea of this jira is to implement a procedure that enables an application to garbage-collect unwanted ledgers.

To garbage-collect a ledger, we need to delete the ledger metadata on ZooKeeper, and delete the ledger data on corresponding bookies.

47799

No Perforce job exists for this issue.

5

33464

22/Mar/10 6:26 PM

9 years, 43 weeks, 2 days ago

0|i063af:

ZooKeeper

ZOOKEEPER-463

C++ tests can't be built on Mac OS using XCode command line tools

Bug

Resolved

Minor

Duplicate

Unassigned

Henry Robinson

13/Jul/09 09:14

23/Apr/14 16:26

3.2.0

3.5.0

tests

0

4

Using latest XCode 3.1.3.
[apache-zookeeper/bin]$ ld -v
@(#)PROGRAM:ld PROJECT:ld64-85.2.1

--wrap is an unsupported command line flag for ld on Mac OS. The cppunit tests therefore won't build.

70793

No Perforce job exists for this issue.

0

33063

23/Apr/14 4:26 PM

5 years, 48 weeks, 1 day ago

0|i060tb:

ZooKeeper

ZOOKEEPER-461

Condense ledger configuration in ZooKeeper

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

11/Jul/09 03:36

01/May/13 22:29

26/Jan/10 18:19

3.3.0

contrib-bookkeeper

0

2

ZOOKEEPER-503, ZOOKEEPER-507

We currently use several znodes to represent the configuration and state of a ledger in ZooKeeper. Although this is good for readability, it makes operations such as create, close, and open more complex, in particular the asynchronous versions of these operations. The main idea of this jira is to store as much as possible of the configuration of a ledger as data of a single znode.

47800

No Perforce job exists for this issue.

2

33465

26/Jan/10 6:19 PM

10 years, 9 weeks, 2 days ago

0|i063an:

ZooKeeper

ZOOKEEPER-460

bad testRetry in cppunit tests (hudson failure)

Bug

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

09/Jul/09 12:39

05/Sep/09 18:36

03/Aug/09 17:35

3.2.1, 3.3.0

c client, tests

0

2

ZOOKEEPER-473

the followng code failed on hudson
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/371/

watchctx_t ctx1, ctx2;
zhandle_t *zk1 = createClient(&ctx1);
CPPUNIT_ASSERT_EQUAL(true, ctx1.waitForConnected(zk1));
zhandle_t *zk2 = createClient(&ctx2);
zookeeper_close(zk1);
CPPUNIT_ASSERT_EQUAL(true, ctx2.waitForConnected(zk2));

there's a problem with this test, it assumes that close(1) can be called before createclient(2) gets connected.

this is not correct: createclient is an async call an in some cases the connection can be established before
create client returns.

this shows a failure in this case because client1 was created, then client2 attempted to connect
but failed due to this on the server (max conn exceeded):
sprintf(cmd, "export ZKMAXCNXNS=1;%s startClean %s", ZKSERVER_CMD, getHostPorts());

conn 2 failed and therefore the following assert eventually failed.

this code should not assume that close(1) will beat connect(2)

Henry can you take a look?

47801

No Perforce job exists for this issue.

1

33064

14/Jul/09 2:36 PM

10 years, 34 weeks, 3 days ago

Reviewed

0|i060tj:

ZooKeeper

ZOOKEEPER-459

Opening a ledger with wrong password doesn't raise an exception

Bug

Resolved

Major

Not A Problem

Flavio Paiva Junqueira

Luca Telloli

03/Jul/09 10:16

18/Dec/10 10:30

3.1.1

contrib-bookkeeper

0

I've experienced by accident the following behavior:
- created a ledger with a password
- opened a ledger with a different password

No exception was raised, the openLedger(...) would return correctly but containing 0 entries

214118

No Perforce job exists for this issue.

0

33065

03/Jul/09 12:13 PM

9 years, 14 weeks, 5 days ago

0|i060tr:

ZooKeeper

ZOOKEEPER-458

connect_index in zookeeper handle might get out of bound.

Bug

Open

Major

Unresolved

Mahadev Konar

02/Jul/09 20:05

14/Dec/19 06:08

3.7.0

c client

0

1

connect_index in zookeeper handle might get out of bound. the zokoeeper_init method checks for index == count and sets it to zero. If the index becomes greater than count, then it will go out of bounds.

68118

No Perforce job exists for this issue.

7

33066

18/Nov/09 9:33 PM

8 years, 21 weeks, 5 days ago

0|i060tz:

ZooKeeper

ZOOKEEPER-457

Make ZookeeperMain public, support for HBase (and other) embedded clients

Bug

Closed

Major

Fixed

ryan rawson

01/Jul/09 14:58

05/Sep/09 18:36

31/Jul/09 16:32

3.1.1

3.2.1, 3.3.0

0

3

hi folks, we have made some changes to zookeeper to facilitate providing an embedded zk client in our own hbase client. This will allow our users to use 1 shell to manipulate both hbase things and zookeeper things. It requires making a few things public, and in the process rearranging how some static things are initialized.

It's fairly trivial refactoring, hopefully you guys approve!

Thanks!

47802

No Perforce job exists for this issue.

1

33067

01/Jul/09 3:07 PM

10 years, 34 weeks, 5 days ago

Reviewed

0|i060u7:

ZooKeeper

ZOOKEEPER-456

CREATOR_ALL_ACL has unnecessary PERMS.ADMIN in the declartion.

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

30/Jun/09 20:30

26/Mar/10 13:24

25/Jan/10 21:04

3.3.0

0

1

CREATOR_ALL_ACL has unnecessary PERMS.ADMIN in the declartion. The perms are defined as PERMS.ALL|PERMS.ADMIN. PERMS.ALL includes ADMIN, so the logical or is unnecessary.

47803

No Perforce job exists for this issue.

1

33466

25/Jan/10 6:18 PM

10 years, 9 weeks, 1 day ago

Reviewed

0|i063av:

ZooKeeper

ZOOKEEPER-455

zookeeper c client crashes with chroot specified in the string.

Bug

Closed

Blocker

Fixed

Mahadev Konar

30/Jun/09 19:44

08/Jul/09 16:24

01/Jul/09 12:39

3.2.0

0

1

the zookeeper c client crashes with chroot specified in the string. This does not fail on 2.6 but on ubuntu where the malloced memory is nto initialized to \0.

47804

No Perforce job exists for this issue.

1

33068

30/Jun/09 8:58 PM

10 years, 39 weeks ago

Reviewed

0|i060uf:

ZooKeeper

ZOOKEEPER-454

allow compilation with jdk1.5

Bug

Closed

Minor

Fixed

Patrick D. Hunt

30/Jun/09 16:53

08/Jul/09 16:24

30/Jun/09 16:57

3.2.0

server

0

1

we no longer officially support jdk1.5 however it still compiles -- except for a recent @override that was added for an interface method. jdk1.6
allows this but jdk1.5 does not (must be superclass). small patch will address this. 3.3 we will drop 1.5 concerns entirely but for now...

47805

No Perforce job exists for this issue.

1

33069

01/Jul/09 6:51 AM

10 years, 39 weeks, 1 day ago

0|i060un:

ZooKeeper

ZOOKEEPER-453

Worker is not removed in QuorumCnxManager upon crash

Bug

Closed

Blocker

Fixed

Flavio Paiva Junqueira

30/Jun/09 12:25

08/Jul/09 16:24

30/Jun/09 16:47

3.2.0

0

1

This causes a problem when we crash and restart a replica R because other replicas believe that they still have a connection to R after restart.

47806

No Perforce job exists for this issue.

4

33070

30/Jun/09 1:09 PM

10 years, 39 weeks, 1 day ago

0|i060uv:

ZooKeeper

ZOOKEEPER-452

zookeeper performance graph should have percentage of reads rather than percentage of writes - zkperfRW-3.2.jpg

Bug

Closed

Major

Fixed

Mahadev Konar

29/Jun/09 21:17

08/Jul/09 16:24

29/Jun/09 21:29

3.2.0

0

1

THe image is wrong and should have percentage of reads on the x axis..

47807

No Perforce job exists for this issue.

1

33071

01/Jul/09 6:51 AM

10 years, 39 weeks, 1 day ago

0|i060v3:

ZooKeeper

ZOOKEEPER-451

ZK should enforce quota

Improvement

Open

Major

Unresolved

Botond Hejj

Raghu S

29/Jun/09 14:47

14/Dec/19 06:07

3.2.0

3.7.0

server

0

8

ZOOKEEPER-2593, ZOOKEEPER-3301

Email exchange with Mahadev:

Mahadev Konar wrote:
> Hi Raghu,
> We do have plans to enforce quota in future. Enforcing requires some more
> work then just reporting. Reporting is a good enough tool for operations to
> manage a zookeeper cluster but we would certainly like to enforce it in the
> near future.
>
> Thanks
> mahadev
>
>
> On 6/18/09 7:01 PM, "raghul@yahoo.com" <raghul@yahoo.com> wrote:
>
>> Is there a reason why node count/byte quota is not actually enforced but
>> rather ZK just warns? Are there any plans to enforce the quota in a future
>> release?
>>
>> Thanks
>> Raghu
>>
>>
>>
>

70785

No Perforce job exists for this issue.

3

68465

29/Jun/09 2:52 PM

1 year, 38 weeks, 1 day ago

0|i0c37r:

ZooKeeper

ZOOKEEPER-450

emphemeral cleanup not happening with session timeout

Bug

Closed

Blocker

Fixed

Benjamin Reed

29/Jun/09 11:29

08/Jul/09 16:24

29/Jun/09 21:32

3.2.0

0

1

The session move patch broke ephemeral cleanup during session expiration. tragically, we didn't have test coverage to detect the bug.

47808

No Perforce job exists for this issue.

1

33072

29/Jun/09 12:43 PM

10 years, 39 weeks, 1 day ago

0|i060vb:

ZooKeeper

ZOOKEEPER-449

sesssionmoved in java code and ZCLOSING in C have the same value.

Bug

Closed

Major

Fixed

Mahadev Konar

26/Jun/09 16:36

08/Jul/09 16:24

26/Jun/09 17:16

3.2.0

0

1

sesssionmoved in java code and ZCLOSING in C have the same value. We need to assign a new value to ZSESSIONMOVED.

47809

No Perforce job exists for this issue.

1

33073

26/Jun/09 5:11 PM

10 years, 39 weeks, 5 days ago

0|i060vj:

ZooKeeper

ZOOKEEPER-448

png files do nto work with forrest.

Bug

Closed

Major

Fixed

Mahadev Konar

26/Jun/09 14:39

08/Jul/09 16:24

26/Jun/09 14:51

3.2.0

0

1

png images are not compatible with forrest generating pdf. We can them to jpg to get them into pdfs.

47810

No Perforce job exists for this issue.

2

33074

26/Jun/09 2:43 PM

10 years, 39 weeks, 5 days ago

0|i060vr:

ZooKeeper

ZOOKEEPER-447

zkServer.sh doesn't allow different config files to be specified on the command line

Improvement

Closed

Minor

Fixed

Henry Robinson

25/Jun/09 06:38

05/Sep/09 18:36

04/Aug/09 17:23

3.1.1, 3.2.0

3.2.1, 3.3.0

0

4

Unless I'm missing something, you can change the directory that the zoo.cfg file is in by setting ZOOCFGDIR but not the name of the file itself.

I find it convenient myself to specify the config file on the command line, but we should also let it be specified by environment variable.

47811

No Perforce job exists for this issue.

1

33467

25/Jun/09 12:11 PM

10 years, 34 weeks, 1 day ago

Reviewed

0|i063b3:

ZooKeeper

ZOOKEEPER-446

some traces of the host auth scheme left

Bug

Closed

Major

Fixed

Benjamin Reed

19/Jun/09 18:44

08/Jul/09 16:24

22/Jun/09 16:55

3.2.0

0

1

the host auth scheme was removed because it used a blocking call in an async pipeline. however, tragically, the blocking call was not removed including a couple of other stray classes.

47812

No Perforce job exists for this issue.

1

33075

20/Jun/09 3:14 AM

10 years, 40 weeks, 2 days ago

0|i060vz:

ZooKeeper

ZOOKEEPER-445

Potential bug in leader code

Bug

Resolved

Minor

Invalid

Unassigned

Manos Kapritsos

18/Jun/09 12:06

23/Apr/14 16:30

3.5.0

server

0

2

1200

0%

Linux fortiz-desktop 2.6.27-7-generic #1 SMP Fri Oct 24 06:42:44 UTC 2008 i686 GNU/Linux
java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode, sharing)

There is a suspicious line in server/quorum/Leader.java:226. It reads
if (stop) {
LOG.info("exception while shutting down acceptor: " + e);
stop = true;
}

0%

1200

70789

No Perforce job exists for this issue.

0

33076

15/Oct/10 9:59 AM

5 years, 48 weeks, 1 day ago

0|i060w7:

ZooKeeper

ZOOKEEPER-444

perms definition for PERMS_ALL differ in C and java

Bug

Closed

Blocker

Fixed

Mahadev Konar

15/Jun/09 14:48

08/Jul/09 16:24

17/Jun/09 13:37

3.1.1

3.2.0

0

1

the perms_all definition in Java is PERMS.ALL and does not include ADMIN perms but in c the PERMS_ALL def includes the ADMIN perms. We should make it consistent to include or not include the admin perms in both c and java.

47813

No Perforce job exists for this issue.

1

33077

17/Jun/09 1:27 AM

10 years, 41 weeks ago

Changed PERMS.ALL definition in ZooDefs in java to include ADMIN. This changes the definition of OPEN_ACL_UNSAFE to have admin perms.

Incompatible change, Reviewed

0|i060wf:

ZooKeeper

ZOOKEEPER-443

trace logging in watch notification not wrapped with istraceneabled - inefficient

Improvement

Closed

Critical

Fixed

Patrick D. Hunt

12/Jun/09 16:24

08/Jul/09 16:24

15/Jun/09 16:52

3.2.0

server

0

In org.apache.zookeeper.server.NIOServerCnxn.process(WatchedEvent) there's a trace message
that's not wrapped with isTraceEnabled, this is very inefficient and should be fixed.

47814

No Perforce job exists for this issue.

1

33468

12/Jun/09 6:22 PM

10 years, 41 weeks, 1 day ago

Reviewed

0|i063bb:

ZooKeeper

ZOOKEEPER-442

ZOOKEEPER-1829 need a way to remove watches that are no longer of interest

Sub-task

Resolved

Critical

Fixed

Rakesh Radhakrishnan

Benjamin Reed

11/Jun/09 10:04

16/Dec/18 09:33

24/Jan/14 01:27

3.5.0

java client, server

10

20

ZOOKEEPER-3053, ZOOKEEPER-1910, ZOOKEEPER-1919, ZOOKEEPER-1887

currently the only way a watch cleared is to trigger it. we need a way to enumerate the outstanding watch objects, find watch events the objects are watching for, and remove interests in an event.

remove_watches

69474

No Perforce job exists for this issue.

16

68466

11/Jun/09 2:27 PM

6 years, 8 weeks, 5 days ago

Reviewed

0|i0c37z:

ZooKeeper

ZOOKEEPER-441

Zk-336 diff got applied twice to TestClientRetry.cc C test, causing compilation failure

Bug

Closed

Blocker

Fixed

Henry Robinson

09/Jun/09 07:32

26/Mar/10 13:31

09/Jun/09 12:46

0

2

The latest version of trunk has a src/c/tests/TestClientRetry.cc file that has the actual file from ZK-336 appended to itself. This causes the compilation to fail due to lots of redeclaration errors.

47815

No Perforce job exists for this issue.

1

33078

09/Jun/09 12:22 PM

10 years, 42 weeks, 2 days ago

Reviewed

0|i060wn:

ZooKeeper

ZOOKEEPER-440

update the performance documentation in forrest

Task

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

08/Jun/09 21:22

08/Jul/09 16:24

29/Jun/09 20:49

3.2.0

documentation

0

Ben, it would be great if you could update the performance documentation in Forrest docs based on the 3.2 performance improvements.
Specifically the scalling graphs (reads vs write load for various quorum sizes)

47816

No Perforce job exists for this issue.

3

33469

29/Jun/09 5:35 PM

10 years, 39 weeks, 1 day ago

0|i063bj:

ZooKeeper

ZOOKEEPER-439

improve error reporting to client during connection establishment

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

08/Jun/09 16:48

14/Dec/19 06:08

3.2.0

3.7.0

c client, java client, server

0

2

ZOOKEEPER-428

If there is a problem on the server during connection est. the server just closes the connection, it would be
better if we could indicate to the client the cause of the failure.

See ZOOKEEPER-428 for background

70732

No Perforce job exists for this issue.

0

68467

10 years, 42 weeks, 3 days ago

0|i0c387:

ZooKeeper

ZOOKEEPER-438

addauth fails to register auth on new client that's not yet connected

Bug

Closed

Blocker

Fixed

Benjamin Reed

Patrick D. Hunt

08/Jun/09 14:04

08/Jul/09 16:24

25/Jun/09 16:42

3.2.0

c client, java client

0

1

if addauth is called on a new client connection that's never connected to the server, when the client does connect
(syncconnected) the auth is not passed to the server. we should ensure we addauth when the client connects or reconnects

47817

No Perforce job exists for this issue.

4

33079

25/Jun/09 1:07 AM

10 years, 39 weeks, 5 days ago

0|i060wv:

ZooKeeper

ZOOKEEPER-437

Variety of Documentation Updates

Improvement

Closed

Trivial

Fixed

Grant Ingersoll

05/Jun/09 23:54

08/Jul/09 16:24

09/Jun/09 19:03

3.2.0

0

Patch with a variety of documentation updates. Some new content for getting started based on my experience trying it out, plus some typo fixes, etc.

47818

No Perforce job exists for this issue.

2

33470

08/Jun/09 5:46 PM

10 years, 42 weeks ago

Reviewed

0|i063br:

ZooKeeper

ZOOKEEPER-436

Bookies should auto register to ZooKeeper

Improvement

Closed

Major

Fixed

Erwin Tam

Benjamin Reed

05/Jun/09 17:12

26/Mar/10 13:24

19/Mar/10 13:13

3.3.0

contrib-bookkeeper

0

2

currently bookies have to be manually added to ZooKeeper to be used in a BookKeeper service. we should be able to just start up a bookkie, point it at ZooKeeper, and have it get auto integrated in.

47819

No Perforce job exists for this issue.

3

33471

16/Mar/10 4:56 PM

10 years, 1 week, 5 days ago

Reviewed

0|i063bz:

ZooKeeper

ZOOKEEPER-435

allow "super" admin digest based auth to be configurable

Bug

Closed

Critical

Fixed

Patrick D. Hunt

05/Jun/09 16:42

08/Jul/09 16:24

09/Jun/09 01:21

3.2.0

server

0

1

the server has a "super" digest based auth user that enables administrative access (ie has access to znodes regardless
of acl settings) but the password is not configurable

1) make the default digest null, ie turn off "super" by default
2) if a command line option is specified when starting server then use the provided digest for super

eg. java -Dzookeeper.DigestAuthenticationProvider.superDigest=xkxkxkxkx ....

also this is not documented in the forrest docs - need to add that along with tests as part of the patch.

47820

No Perforce job exists for this issue.

1

33080

05/Jun/09 7:41 PM

10 years, 42 weeks ago

Reviewed

0|i060x3:

ZooKeeper

ZOOKEEPER-434

the java shell should indicate connection status on command prompt

Improvement

Closed

Minor

Fixed

Henry Robinson

Patrick D. Hunt

05/Jun/09 14:01

08/Jul/09 16:24

09/Jun/09 01:10

3.1.1

3.2.0

java client

1

it would be very useful if the java shell showed the current connection status as part of the command prompt.

this shows itself in particular for the following use case:

I attempted to connect a java shell to a remote cluster that was unavailable, when I run the first command "ls /" on
the cluster the shell hangs. It would be nice if the shell indicated connection status in the prompt and make it more
clear that the shell is currently not connected. (it was hard to see the "attempting to connect" console message as
it was lost in with the other messaes...)

47821

No Perforce job exists for this issue.

1

33472

06/Jun/09 7:17 AM

10 years, 42 weeks ago

Reviewed

0|i063c7:

ZooKeeper

ZOOKEEPER-433

getacl on root znode (/) fails

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

05/Jun/09 13:57

08/Jul/09 16:24

17/Jun/09 21:09

3.1.1

3.2.0

server

0

1

running a "getAcl" on the root znode "/" fails with the following:

Fri Jun 5 10:21:17 2009: 2009-06-05 10:21:17,072 - ERROR [CommitProcessor:3:FinalRequestProcessor@243] - Failed to process sessionid:0x321b16868f40003 ty
pe:getACL cxid:0x3 zxid:0xfffffffffffffffe txntype:unknown n/a
Fri Jun 5 10:21:17 2009: java.lang.NullPointerException
Fri Jun 5 10:21:17 2009: at java.util.ArrayList.<init>(ArrayList.java:131)
Fri Jun 5 10:21:17 2009: at org.apache.zookeeper.server.DataTree.getACL(DataTree.java:622)
Fri Jun 5 10:21:17 2009: at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:216)
Fri Jun 5 10:21:17 2009: at org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:74)
Fri Jun 5 10:21:17 2009: 2009-06-05 10:21:17,073 - ERROR [CommitProcessor:3:FinalRequestProcessor@250] - Dumping request buffer: 0x00012f

We need to support getting/setting the root acl in particular -- not being able to control acls on this node makes multi-tenancy a non-starter.

47822

No Perforce job exists for this issue.

3

33081

09/Jun/09 12:05 AM

10 years, 41 weeks ago

fixes a problem with getacl on root node "/"

Reviewed

0|i060xb:

ZooKeeper

ZOOKEEPER-432

Various improvements to zkpython bindings

Improvement

Closed

Major

Fixed

Henry Robinson

05/Jun/09 08:09

08/Jul/09 16:24

15/Jun/09 18:54

3.2.0

contrib-bindings

0

2

What's new, from the README for this new version:

----------------------------------
More test coverage.

Better reference counting, fixing at least two serious bugs.

Out-of-range zhandles are now checked, fixing a potential security hole.

Docstrings! Editing and cleanup required, but most of the text is there.

zookeeper.set_watcher is now implemented correctly.

zookeeper.client_id is now implemented correctly. zookeeper.init now respects the client_id parameter.

get_context and set_context have been removed from the API. The context mechanism is used by PyZK to store the callables that are dispatched by C-side watchers. Messing with this from Python-side causes bugs very quickly. You should wrap all desired context up in a callable and then use zookeeper.set_watcher to attach it to the global watcher.

Many methods now have optional parameters (usually if you can specify a watch, it's optional). The only time where genuinely optional parameters are still mandatory is when a required parameters comes after it. Currently we still respect the ZK C client parameter ordering. For example, you can simply connect with zookeeper.init("host:port") and ignore the other three parameters.

47823

No Perforce job exists for this issue.

2

33473

08/Jun/09 5:48 PM

10 years, 41 weeks, 1 day ago

Reviewed

0|i063cf:

ZooKeeper

ZOOKEEPER-431

Expose methods to ease ZK integration

Improvement

Closed

Major

Fixed

Jean-Daniel Cryans

03/Jun/09 16:05

08/Jul/09 16:24

05/Jun/09 00:45

3.2.0

2

1

HBASE-1449

HBase committer here. I opened this jira to see if it's possible to include some changes in the way quorum servers are started in order to ease the integration with HBase. Basically the major things we would change is how methods are exposed in QPM/QPC and such.

47824

No Perforce job exists for this issue.

1

33474

03/Jun/09 4:15 PM

10 years, 42 weeks, 6 days ago

Reviewed

0|i063cn:

ZooKeeper

ZOOKEEPER-430

improve error reporting to client when setdata data length > max (default 1mb)

Bug

Open

Minor

Unresolved

Unassigned

Patrick D. Hunt

03/Jun/09 12:41

14/Dec/19 06:09

3.7.0

c client, java client, server

0

currently when a node is set > 1mb (default) the server disconnects the client and logs information about node too large.
however the client has no indication why the connection was killed. we need to improve the error reporting here.
(might be a good time to review error reporting in general)

70769

No Perforce job exists for this issue.

0

33082

10 years, 43 weeks, 1 day ago

0|i060xj:

ZooKeeper

ZOOKEEPER-429

Python contrib test failing on hudson

Bug

Resolved

Major

Invalid

Unassigned

Flavio Paiva Junqueira

03/Jun/09 03:14

03/Jun/09 07:31

0

I was checking the console output of:

http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/100/

and I observed the following output for the contrib tests:
{noformat}
[exec] ======================================================================
[exec] ======================================================================
[exec] Running contrib tests.
[exec] ======================================================================
[exec] ======================================================================
[exec]
[exec]
[exec] Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
[exec] /bin/kill -9 30147
[exec] kill: No such process
[exec] /home/hudson/tools/ant/latest/bin/ant -Declipse.home=/home/nigel/tools/eclipse/latest -Dpython.home=/home/nigel/tools/python/latest -DZooKeeperPatchProcess= -Dtest.junit.output.format=xml -Dtest.output=yes test-contrib
[exec] Buildfile: build.xml
[exec]
[exec] test-contrib:
[exec]
[exec] BUILD SUCCESSFUL
[exec] Total time: 0 seconds
{noformat}

214117

No Perforce job exists for this issue.

0

33083

03/Jun/09 7:09 AM

10 years, 43 weeks, 1 day ago

0|i060xr:

ZooKeeper

ZOOKEEPER-428

logging should be makred as warn rathen than error in NIOServerCnxn.

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

02/Jun/09 17:21

08/Jul/09 16:24

15/Jun/09 19:00

3.2.0

0

ZOOKEEPER-439

Errors such as these should be marked as warn and not error.

7:54,094 ERROR [NIOServerCxn.Factory:21810]
server.NIOServerCnxn(543): Client has seen zxid 0x10 our last zxid is 0x4

47825

No Perforce job exists for this issue.

1

33475

08/Jun/09 4:49 PM

10 years, 41 weeks, 1 day ago

Reviewed

0|i063cv:

ZooKeeper

ZOOKEEPER-427

ZooKeeper server unexpectedly high CPU utilisation

Bug

Closed

Blocker

Fixed

Flavio Paiva Junqueira

Satish Kotha

02/Jun/09 17:01

11/Oct/10 03:34

18/Jun/09 16:33

3.1.1

3.2.0

0

1

DIRMINA-678, ZOOKEEPER-287, ZOOKEEPER-893

Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode)

I am running a 5 node ZooKeeper cluster and I noticed that one of them has very high CPU usage:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6883 infact 22 0 725m 41m 4188 S 95 0.5 5671:54 java

It is not "doing anything" application-wise at this point, so I was wondering why the heck it's using up so much CPU.

47826

No Perforce job exists for this issue.

5

33084

02/Jun/09 5:16 PM

10 years, 40 weeks, 3 days ago

Reviewed

0|i060xz:

ZooKeeper

ZOOKEEPER-426

Windows versions of zookeeper scripts

Improvement

Closed

Major

Fixed

David Bosschaert

02/Jun/09 10:39

26/Mar/10 13:24

05/Jan/10 12:39

3.1.1

3.3.0

scripts

1

4

ZOOKEEPER-606

Attached are a some Windows versions of the zookeeper scripts. They aren't as powerful as the .sh ones but they do work for me and can be invoked with any directory as the current dir.
The biggest trick in the scripts is in the zkEnv.cmd one where it says:
set ZOOCFGDIR=%~dp0%..\conf
this basically figures out the location of the zkEnv.cmd file and sets the conf directory relative to that.

47827

No Perforce job exists for this issue.

7

33476

03/Dec/09 5:44 PM

10 years, 12 weeks, 1 day ago

Reviewed

0|i063d3:

ZooKeeper

ZOOKEEPER-425

Add OSGi metadata to zookeeper.jar

Improvement

Closed

Major

Fixed

David Bosschaert

02/Jun/09 10:28

01/May/13 22:29

26/Nov/09 08:37

3.3.0

build

6

4

ZOOKEEPER-749, ZOOKEEPER-584

After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi bundle as well as an ordinary jar file.
In the CXF/DOSGi project the buildsystem does this using the maven-bundle-plugin: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml

The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, this works for the CXF/DOSGi project.
If your buildsystem isn't using maven, I would advise to use bnd (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you should be able to use more or less the same instructions as were used in maven:
<instructions>
<Bundle-Name>ZooKeeper bundle</Bundle-Name>
<Bundle-Description>This bundle contains the ZooKeeper library</Bundle-Description>
<Bundle-SymbolicName>org.apache.hadoop.zookeeper</Bundle-SymbolicName>
<Bundle-Version>3.1.1</Bundle-Version>
<Import-Package>*</Import-Package>
<Export-Package>*;version=3.1.1</Export-Package>
</instructions>

Oh and one other thing. Is it really necessary to put the source code in the Jar file too? I would put that in a separate source distribution :)

See also: http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4A2009B1.3030603@yahoo-inc.com%3e

47828

No Perforce job exists for this issue.

3

33477

11/Nov/09 12:58 PM

9 years, 49 weeks ago

Reviewed

osgi

0|i063db:

ZooKeeper

ZOOKEEPER-424

server side chroot enforcment - link to auth

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

01/Jun/09 16:40

14/Dec/19 06:07

3.7.0

server

0

ZOOKEEPER-237

Allow the server administrator to enforce a particular root on specific authenticated users.

ZOOKEEPER-237 implements the client side of this - the client can set a chroot, however this doesn't allow
someone like an administrator to enforce the root.

We should add the ability to the server to verify that all accesses are to a particular root.

We currently have ACLs which provide essentially this, however there are a few small issues where root enforcement
would be useful from server operator perspective.

70758

No Perforce job exists for this issue.

0

68468

21/Feb/11 8:11 AM

9 years, 5 weeks, 3 days ago

0|i0c38f:

ZooKeeper

ZOOKEEPER-423

Add getFirstChild API

New Feature

Open

Major

Unresolved

Unassigned

Henry Robinson

01/Jun/09 12:10

29/Apr/11 12:07

contrib-bindings, documentation, java client, server

4

9

ZOOKEEPER-1022, ZOOKEEPER-282

When building the distributed queue for my tutorial blog post, it was pointed out to me that there's a serious inefficiency here.

Informally, the items in the queue are created as sequential nodes. For a 'dequeue' call, all items are retrieved and sorted by name by the client in order to find the name of the next item to try and take. This costs O( n ) bandwidth and O(n.log n) sorting time - per dequeue call! Clearly this doesn't scale very well.

If the servers were able to maintain a data structure that allowed them to efficiently retrieve the children of a node in order of the zxid that created them this would make successful dequeue operations O( 1 ) at the cost of O( n ) memory on the server (to maintain, e.g. a singly-linked list as a queue). This is a win if it is generally true that clients only want the first child in creation order, rather than the whole set.

We could expose this to the client via this API: getFirstChild(handle, path, name_buffer, watcher) which would have much the same semantics as getChildren, but only return one znode name.

Sequential nodes would still allow the ordering of znodes to be made explicitly available to the client in one RPC should it need it. Although: since this ordering would now be available cheaply for every set of children, it's not completely clear that there would be that many use cases left for sequential nodes if this API was augmented with a getChildrenInCreationOrder call. However, that's for a different discussion.

A halfway-house alternative with more flexibility is to add an 'order' parameter to getFirstChild and have the server compute the first child according to the requested order (creation time, update time, lexicographical order). This saves bandwidth at the expense of increased server load, although servers can be implemented to spend memory on pre-computing commonly requested orders. I am only in favour of this approach if servers maintain a data-structure for every possible order, and then the memory implications need careful consideration.

[edit - JIRA interprets ( n ) without the spaces as a thumbs-down. cute.]

214116

No Perforce job exists for this issue.

1

68469

02/Jun/09 11:48 AM

8 years, 47 weeks, 6 days ago

0|i0c38n:

ZooKeeper

ZOOKEEPER-422

Java CLI should support ephemeral and sequential node creation

Improvement

Closed

Minor

Fixed

Henry Robinson

30/May/09 09:07

08/Jul/09 16:24

18/Jun/09 18:23

3.2.0

1

The C client supports creation of ephemeral and sequential nodes. For feature parity, so should the Java CLI.

47829

No Perforce job exists for this issue.

1

33478

11/Jun/09 8:32 PM

10 years, 40 weeks, 3 days ago

0|i063dj:

ZooKeeper

ZOOKEEPER-421

zkpython run_tests.sh is missing #!

Bug

Closed

Minor

Fixed

Henry Robinson

Patrick D. Hunt

28/May/09 14:39

08/Jul/09 16:24

01/Jun/09 17:00

3.2.0

contrib-bindings

0

1

the scripts in the test directory of zkpython are missing #! headers

Probably:

#!/bin/sh

for shell scripts and

#!/usr/bin/python

for .py scripts?

Also include a shell script that will svn chmod the *.py scripts so that they can be executed individually from the command line (shortcut
rather than (python foo.py).

47830

No Perforce job exists for this issue.

2

33085

30/May/09 9:30 AM

10 years, 43 weeks, 2 days ago

Reviewed

0|i060y7:

ZooKeeper

ZOOKEEPER-420

build/test should not require install in zkpython

Bug

Closed

Major

Fixed

Henry Robinson

Patrick D. Hunt

28/May/09 14:36

26/Mar/10 13:24

01/Oct/09 17:38

3.2.2, 3.3.0

contrib-bindings

0

4

Currently you cannot just build and test the zkpython contrib, you need to actually install the zookeeper client c library as well
as the zkpython lib itself.

There really needs to be 2 steps:

1) build/test zkpython "encapsulated" within the src repository, there should be no requirement to actually install anything
(this is esp the case for automated processes and for review by PMC during release time for example)
2) build an egg that can be distributed/installed by end user

47831

No Perforce job exists for this issue.

6

33086

09/Jun/09 9:46 AM

10 years, 25 weeks, 6 days ago

Reviewed

0|i060yf:

ZooKeeper

ZOOKEEPER-419

Reference counting bug in Python bindings causes abort errors

Bug

Closed

Critical

Fixed

Henry Robinson

27/May/09 20:24

08/Jul/09 16:24

28/May/09 14:31

3.2.0

contrib-bindings

0

3

Due to reference counts being incremented incorrectly for stat-based calls, Python's GC occasionally aborts.

47832

No Perforce job exists for this issue.

1

33087

27/May/09 11:31 PM

10 years, 43 weeks, 6 days ago

Reviewed

0|i060yn:

ZooKeeper

ZOOKEEPER-418

Need nifty zookeeper browser

Bug

Closed

Major

Won't Fix

Ted Dunning

27/May/09 03:10

23/Nov/11 14:22

12/Dec/10 20:03

3.4.0

2

4

ZOOKEEPER-678

It would be very nice to have a browser that would allow the state of a Zoo to be examined. Even nice would be such a utility that showed changes in real time.

214115

No Perforce job exists for this issue.

4

33088

27/May/09 4:27 PM

9 years, 15 weeks, 3 days ago

0|i060yv:

ZooKeeper

ZOOKEEPER-417

stray message problem when changing servers

Bug

Closed

Blocker

Fixed

Benjamin Reed

26/May/09 14:10

08/Jul/09 16:24

26/Jun/09 17:09

3.2.0

0

1

There is a possibility for stray messages from a previous connection to violate ordering and generally cause problems. Here is a scenario: we have a client, C, two followers, F1 and F2, and a leader, L. The client is connected to F1, which is a slow follower. C sends setData("/a", "1") to F1 and then loses the connection, so C reconnects to F2 and sends setData("/a", "2"). it is possible, if F1 is slow enough and the setData("/a", "1") got onto the network before the connection break, for F1 to forward the setData("/a", "1") to L after F2 forwards setData("/a", "2").

to fix this, the leader should keep track of which follower last registered a session for a client and drop any requests from followers for clients for whom they do not have a registration.

47833

No Perforce job exists for this issue.

4

33089

24/Jun/09 3:39 AM

10 years, 39 weeks, 5 days ago

0|i060z3:

ZooKeeper

ZOOKEEPER-416

BookKeeper jar includes unnecessary files

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

26/May/09 11:51

08/Jul/09 16:24

28/May/09 00:55

3.2.0

contrib-bookkeeper

0

1

I was checking the bookkeper jar, and I found that it includes some unnecessary files related to junit, such as:

{noformat}
0 Tue May 12 19:00:14 PDT 2009 tmp/
0 Tue May 12 19:00:00 PDT 2009 tmp/test14667.junit.dir/
0 Tue May 12 19:00:08 PDT 2009 tmp/test14667.junit.dir/version-2/
0 Tue May 12 19:00:10 PDT 2009 tmp/test16109.junit.dir/
0 Tue May 12 19:00:16 PDT 2009 tmp/test16109.junit.dir/version-2/
0 Tue May 12 19:00:14 PDT 2009 tmp/test16113.junit.dir/
0 Tue May 12 19:00:16 PDT 2009 tmp/test16113.junit.dir/version-2/
2256 Tue May 12 18:59:08 PDT 2009 logs/TEST-org.apache.bookkeeper.test.BookieClientTest.txt
167046 Tue May 12 19:00:00 PDT 2009 logs/TEST-org.apache.bookkeeper.test.BookieReadWriteTest.txt
13035 Tue May 12 19:00:08 PDT 2009 logs/TEST-org.apache.bookkeeper.test.CloseTest.txt
25036 Tue May 12 19:00:16 PDT 2009 logs/TEST-org.apache.bookkeeper.test.LedgerRecoveryTest.txt
896 Tue May 12 19:00:16 PDT 2009 logs/TEST-org.apache.bookkeeper.test.NIOServerFactoryTest.txt
{noformat}

47834

No Perforce job exists for this issue.

1

33090

26/May/09 8:36 PM

10 years, 44 weeks ago

Reviewed

0|i060zb:

ZooKeeper

ZOOKEEPER-415

zookeeper c tests hang.

Bug

Closed

Major

Fixed

Mahadev Konar

21/May/09 16:29

08/Jul/09 16:24

22/May/09 03:25

3.2.0

0

2

the c tests hang sometimes.

47835

No Perforce job exists for this issue.

1

33091

21/May/09 9:57 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i060zj:

ZooKeeper

ZOOKEEPER-414

zkServer.sh is duplicated in a couple of places

Improvement

Open

Major

Unresolved

Mahadev Konar

Benjamin Reed

20/May/09 18:06

14/Dec/19 06:08

3.7.0

0

1

we have zkServer.sh in 4 different places. some differ only in whitespace. it would be much more maintainable if there were only one version that everything used.

70726

No Perforce job exists for this issue.

0

68470

20/May/09 6:30 PM

10 years, 19 weeks, 6 days ago

0|i0c38v:

ZooKeeper

ZOOKEEPER-413

two flaws need addressing in the c tests that can cause false positive failures

Bug

Closed

Minor

Fixed

Patrick D. Hunt

20/May/09 17:59

26/Mar/10 13:24

25/Jan/10 21:57

3.3.0

c client, tests

0

2

1) createClient in testclient.cc (check all tests) is not correctly waiting for syncconnected to the server
2) there are some instances of while(xxx); in the test code, this could cause problems, really we need to
have some limit on the number of iterations (other than just the test, which may never return false), also the
loop should have some sort of sleep(100msec) (whatever time) in order to limit cpu use.

47836

No Perforce job exists for this issue.

1

33092

17/Jun/09 6:48 PM

10 years, 9 weeks, 1 day ago

Reviewed

0|i060zr:

ZooKeeper

ZOOKEEPER-412

checkstyle target fails trunk build

Bug

Closed

Major

Fixed

Akihiro Suda

Giridharan Kesavan

20/May/09 04:35

21/Jul/16 16:18

15/Dec/15 14:17

3.5.2

0

7

BUILD FAILED
/home/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/build.xml:865: Unable to create a Checker: cannot initialize module PackageHtml - Unable to instantiate PackageHtml
Tnx!

62830

No Perforce job exists for this issue.

4

33093

02/Sep/11 1:55 PM

4 years, 14 weeks, 1 day ago

Reviewed

0|i060zz:

ZooKeeper

ZOOKEEPER-411

Building zookeeper fails on RHEL 5 64 bit during test-cppunit

Bug

Closed

Major

Fixed

Mahadev Konar

Lee Tucker

19/May/09 18:39

08/Jul/09 16:24

20/May/09 18:41

3.2.0

0

1

RHEL5, 64bit

[exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset : assertion
[exec]
[exec] /grid/0/gs/gridre/hudson/workspace/zootestbuild/trunk/src/c/tests/TestClient.cc:499: Assertion: assertion failed [Expression: ctx.waitForDisconnected(zk)]
[exec] Failures !!!
[exec] Run: 32 Failure total: 1 Failures: 1 Errors: 0
[exec] make: *** [run-check] Error 1

47837

No Perforce job exists for this issue.

4

33094

20/May/09 4:40 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i06107:

ZooKeeper

ZOOKEEPER-410

ZOOKEEPER-406 address all findbugs warnings in client/server classes

Sub-task

Closed

Major

Fixed

Patrick D. Hunt

18/May/09 14:16

08/Jul/09 16:24

20/May/09 14:48

3.2.0

0

trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java
trunk/src/java/main/org/apache/zookeeper/ZooKeeperMain.java
trunk/src/java/main/org/apache/zookeeper/server/SessionTrackerImpl.java
trunk/src/java/main/org/apache/zookeeper/server/SyncRequestProcessor.java
trunk/src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java
trunk/src/java/main/org/apache/zookeeper/server/PrepRequestProcessor.java
trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServer.java

47838

No Perforce job exists for this issue.

1

33479

20/May/09 1:35 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i063dr:

ZooKeeper

ZOOKEEPER-409

ZOOKEEPER-406 address all findbugs warnings in jute related classes

Sub-task

Closed

Major

Fixed

Patrick D. Hunt

18/May/09 14:15

08/Jul/09 16:24

22/May/09 15:06

3.2.0

0

trunk/src/java/main/org/apache/jute/XmlInputArchive.java
trunk/src/java/main/org/apache/jute/compiler/JFile.java
trunk/src/java/main/org/apache/jute/compiler/JRecord.java
trunk/src/java/main/org/apache/jute/compiler/CGenerator.java
trunk/src/java/main/org/apache/jute/compiler/CppGenerator.java
trunk/src/java/main/org/apache/jute/compiler/JavaGenerator.java
org/apache/zookeeper/proto/AuthPacket.java
org/apache/zookeeper/proto/ConnectRequest.java
org/apache/zookeeper/proto/ConnectResponse.java
org/apache/zookeeper/proto/CreateRequest.java
org/apache/zookeeper/proto/GetChildrenResponse.java
org/apache/zookeeper/proto/GetDataResponse.java
org/apache/zookeeper/proto/SetDataRequest.java
org/apache/zookeeper/proto/op_result_t.java
org/apache/zookeeper/server/quorum/QuorumPacket.java
org/apache/zookeeper/txn/CreateTxn.java
org/apache/zookeeper/txn/SetDataTxn.java

47839

No Perforce job exists for this issue.

1

33480

21/May/09 9:10 PM

10 years, 44 weeks, 5 days ago

Reviewed

0|i063dz:

ZooKeeper

ZOOKEEPER-408

ZOOKEEPER-406 address all findbugs warnings in persistence classes

Sub-task

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

18/May/09 14:14

08/Jul/09 16:24

18/Jun/09 12:39

3.2.0

0

trunk/src/java/main/org/apache/zookeeper/server/DataTree.java
trunk/src/java/main/org/apache/zookeeper/server/persistence/FileTxnSnapLog.java
trunk/src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java
trunk/src/java/main/org/apache/zookeeper/server/persistence/Util.java
trunk/src/java/main/org/apache/zookeeper/server/DataNode.java
trunk/src/java/main/org/apache/zookeeper/server/upgrade/DataNodeV1.java
trunk/src/java/main/org/apache/zookeeper/server/upgrade/DataTreeV1.java

47840

No Perforce job exists for this issue.

6

33481

01/Jun/09 10:26 PM

10 years, 40 weeks, 3 days ago

Reviewed

0|i063e7:

ZooKeeper

ZOOKEEPER-407

ZOOKEEPER-406 address all findbugs warnings in org.apache.zookeeper.server.quorum.** packages

Sub-task

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

18/May/09 13:50

08/Jul/09 16:24

20/May/09 16:17

3.2.0

server

0

Flavio please address the findbugs warnings in quorum and sub packages. See
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/308/violations/
for a list of issues or use ant's findbugs (or eclipse) to identify the issues.
In general we should try to fix these, but ignoring the warning (annotation) is ok if it includes comments to effect why, also has
to be reviewed/approved.

47841

No Perforce job exists for this issue.

3

33482

19/May/09 11:19 AM

10 years, 44 weeks, 6 days ago

Reviewed

0|i063ef:

ZooKeeper

ZOOKEEPER-406

address all findbugs warnings in mainline java code

Improvement

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

18/May/09 13:47

08/Jul/09 16:24

18/Jun/09 12:39

3.2.0

java client, server

0

ZOOKEEPER-407, ZOOKEEPER-408, ZOOKEEPER-409, ZOOKEEPER-410

address; fix or mark invalid (ie ignore with comment to the effect why) all findbugs warnings in the mainline org.apache.zookeeper package and
all subpackages. This is in src/java/main/org/apache/zookeeper/**

see for an example:
http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/ZooKeeper-trunk/308/violations/

you can run findbugs in either eclipse or using the findbugs ant target

47842

No Perforce job exists for this issue.

0

33483

03/Jun/09 9:20 AM

10 years, 41 weeks ago

0|i063en:

ZooKeeper

ZOOKEEPER-405

nullpointer exception in zookeeper java shell.

Bug

Closed

Major

Fixed

Mahadev Konar

18/May/09 13:24

08/Jul/09 16:24

20/May/09 13:59

3.2.0

0

1

The java client shell does not handle null return data and throws out null pointer exception.

47843

No Perforce job exists for this issue.

1

33095

20/May/09 1:12 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i0610f:

ZooKeeper

ZOOKEEPER-404

nightly build failed on hudson.

Bug

Closed

Critical

Fixed

Henry Robinson

Mahadev Konar

18/May/09 12:50

08/Jul/09 16:24

19/May/09 17:27

3.2.0

0

1

the nightly build failed with the following error

compile:
[echo] contrib: zkpython

BUILD FAILED
/home/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/build.xml:444: The following error occurred while executing this line:
/home/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/src/contrib/build.xml:39: The following error occurred while executing this line:
/home/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/src/contrib/build-contrib.xml:79: srcdir "/home/hudson/hudson-slave/workspace/ZooKeeper-trunk/trunk/src/contrib/zkpython/src/java" does not exist!

Total time: 32 seconds
Publishing Javadoc
Recording test results
Recording fingerprints
Publishing Clover coverage report...
Sending e-mails to: zookeeper-dev@hadoop.apache.org
finished: FAILURE

47844

No Perforce job exists for this issue.

2

33096

18/May/09 3:01 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i0610n:

ZooKeeper

ZOOKEEPER-403

cleanup javac compiler warnings

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

14/May/09 14:33

08/Jul/09 16:24

20/May/09 14:51

3.2.0

java client, server

0

Cleanup javac warnings, also turn on lint by default in ant build.xml for main org.apache.zookeeper classes.

47845

No Perforce job exists for this issue.

2

33484

14/May/09 5:28 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i063ev:

ZooKeeper

ZOOKEEPER-402

zookeeper c library segfaults on data for a node in zookeeper being null.

Bug

Closed

Critical

Fixed

Mahadev Konar

13/May/09 18:36

08/Jul/09 16:24

21/May/09 17:22

3.1.0, 3.1.1

3.2.0

0

2

the zookeeper c client library seg faults on data being null for a zoo node.

47846

No Perforce job exists for this issue.

3

33097

20/May/09 1:14 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i0610v:

ZooKeeper

ZOOKEEPER-401

Cleaning of logging in the code and log4j configuration might be a good idea

Bug

Open

Minor

Unresolved

Unassigned

Konstantin I Boudnik

12/May/09 17:35

0

During the conversion of mainline tests from JUnit into TestNG it has been noticed (partially because TestNG doesn't intercept tests output) that some of log4j configuration parameters aren't very handy. E.g.
- some of really non-important messages go at WARN level
- by default all output goes into ConsoleAppender, which is a lot and makes it difficult to trace the execution from simply observing the console output
- it'd be good to have a couple different log4j configs to be able to turn ConsoleAppender on and off by simply passing a parameter to ant build

214114

No Perforce job exists for this issue.

0

33098

10 years, 46 weeks, 2 days ago

0|i06113:

ZooKeeper

ZOOKEEPER-400

Issues with procedure to close ledger

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

12/May/09 17:26

01/May/13 22:29

14/May/09 04:22

3.2.0

contrib-bookkeeper

0

1

ZOOKEEPER-383

When running a few experiments with BookKeeper, I found a couple of issues with closing a ledger:

* On ClientCBWorker::run(), we should call setAddConfirmed before invoking the callback method. Otherwise, it is possible that an application closes a ledger before ClientCBWorker modifies the last confirmed operation, and the value written to ZooKeeper won't be the last one written;
* LedgerHandle should write the last add confirmed instead of the last counter. The last attribute counts the operations issued, and we use it to determine the id of the next entry. If an application calls close upon receiving all callbacks, then with the previous modification, the last confirmed add must be equal to (last-1). However, if an application invokes close before receiving all callbacks, the ledger may be left in an inconsistent state because the last entry written to ZooKeeper may be an operation that hasn't completed yet.

Although the modifications required are simple, they are important.

47847

No Perforce job exists for this issue.

1

33099

12/May/09 6:44 PM

10 years, 46 weeks ago

0|i0611b:

ZooKeeper

ZOOKEEPER-399

ZOOKEEPER-394 system tests conversion

Sub-task

Resolved

Major

Won't Fix

Konstantin I Boudnik

09/May/09 18:58

22/Sep/09 19:36

0

214113

No Perforce job exists for this issue.

0

33485

10 years, 46 weeks, 5 days ago

0|i063f3:

ZooKeeper

ZOOKEEPER-398

ZOOKEEPER-394 contrib tests convertion

Sub-task

Resolved

Major

Won't Fix

Konstantin I Boudnik

09/May/09 18:57

22/Sep/09 19:36

contrib

0

In this phase conversion of src/contrib/**/test will happen

214112

No Perforce job exists for this issue.

0

33486

10 years, 46 weeks, 5 days ago

0|i063fb:

ZooKeeper

ZOOKEEPER-397

ZOOKEEPER-394 mainline tests conversion

Sub-task

Closed

Major

Won't Fix

Konstantin I Boudnik

09/May/09 18:57

26/Mar/10 13:24

22/Sep/09 19:36

3.3.0

tests

0

1

In this stage main set (src/java/test) of ZK tests will be converted to TestNG

214111

No Perforce job exists for this issue.

9

33487

12/May/09 6:25 PM

10 years, 38 weeks, 1 day ago

0|i063fj:

ZooKeeper

ZOOKEEPER-396

race condition in zookeeper client library between zookeeper_close and zoo_synchronous api

Bug

Closed

Critical

Fixed

Mahadev Konar

08/May/09 20:41

08/Jul/09 16:24

08/Jun/09 16:39

3.1.1

3.2.0

c client

0

1

There is a race condition in zoopkeeper client library wherein if the application calls zookeeper_close() and zoo_anysynchronouscall simultaneously, sometimes the zoo_sync api call gets hung waiting for a notification whcih will never come.
We might want to create another bugfix release for this.

47848

No Perforce job exists for this issue.

2

33100

04/Jun/09 1:28 PM

10 years, 42 weeks ago

Reviewed

0|i0611j:

ZooKeeper

ZOOKEEPER-395

Python bindings

New Feature

Closed

Major

Fixed

Henry Robinson

08/May/09 19:59

21/May/12 03:17

14/May/09 02:38

3.2.0

0

2

ZOOKEEPER-765

ZooKeeper doesn't have Python bindings. Having them would be useful, and would complement the extant Perl bindings.

47849

No Perforce job exists for this issue.

5

33488

08/May/09 8:29 PM

10 years, 46 weeks ago

Python bindings for ZooKeeper. See src/contrib/zkpython/README for details.

Reviewed

0|i063fr:

ZooKeeper

ZOOKEEPER-394

Converting JUnit tests into TestNG controlled environment

Test

Closed

Major

Invalid

Konstantin I Boudnik

08/May/09 14:33

26/Mar/10 13:24

16/Sep/09 14:30

3.1.1

3.3.0

tests

0

1

ZOOKEEPER-397, ZOOKEEPER-398, ZOOKEEPER-399

TestNG is a powerful test harness, which provides a lot of useful features like parallel test execution, test parametrization, and such.
ZooKeeper uses JUnit (v.4.4) environment instead which certainly should help to ease the conversion

I'd suggest to convert ZooKeeper's Junit based test infrastructure into TestNG controller environment, which will allow to achieve higher level of test execution control in the long run.

The same conversion has been already performed for Avro project (see https://issues.apache.org/jira/browse/AVRO-26)

214110

No Perforce job exists for this issue.

0

33489

19/May/09 6:09 PM

10 years, 28 weeks, 1 day ago

0|i063fz:

ZooKeeper

ZOOKEEPER-392

Change log4j properties

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

06/May/09 17:37

08/Jul/09 16:24

07/May/09 18:28

3.1.1

3.2.0

0

Change logging level to info in conf/log4j.properties.

47850

No Perforce job exists for this issue.

1

33490

07/May/09 6:28 PM

10 years, 46 weeks, 6 days ago

Reviewed

0|i063g7:

ZooKeeper

ZOOKEEPER-391

bookeeper mainline code should not be calling printStackTrace

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

06/May/09 14:25

08/Jul/09 16:24

12/May/09 17:00

3.2.0

contrib-bookkeeper

0

1

Bookeeper mainline code is calling printStackTrace, it should be using logging instead.

47851

No Perforce job exists for this issue.

1

33101

11/May/09 8:51 AM

10 years, 46 weeks, 1 day ago

Reviewed

0|i0611r:

ZooKeeper

ZOOKEEPER-390

zookeeper url scheme

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

05/May/09 12:53

28/Nov/16 06:33

1

6

ZOOKEEPER-1355, ZOOKEEPER-237, ZOOKEEPER-146, ZOOKEEPER-107

we need a URL scheme for zookeeper. i think such a scheme could encode various forms of server lists as well as chroot.

214109

No Perforce job exists for this issue.

0

68471

28/Nov/16 6:33 AM

3 years, 16 weeks, 3 days ago

0|i0c393:

ZooKeeper

ZOOKEEPER-389

add help/usage to the c shell cli.c

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

01/May/09 16:47

08/Jul/09 16:24

07/May/09 18:31

3.2.0

c client

0

The current c shell should have help information (in the shell)

currently it's hard to know what commands are available

47852

No Perforce job exists for this issue.

1

33491

01/May/09 6:45 PM

10 years, 46 weeks, 6 days ago

Reviewed

0|i063gf:

ZooKeeper

ZOOKEEPER-388

Add readline support to the c shell

Improvement

Open

Minor

Unresolved

Unassigned

Patrick D. Hunt

01/May/09 16:46

14/Dec/19 06:07

3.7.0

c client

0

ZOOKEEPER-386

Add readline support to the c shell

autotools should detect readline libs and enable this by default if present (should also support turning off).

70788

No Perforce job exists for this issue.

0

68472

10 years, 45 weeks, 2 days ago

0|i0c39b:

ZooKeeper

ZOOKEEPER-387

closeLedger shouldn't shut down all QuorumEngines

Bug

Resolved

Major

Duplicate

Flavio Paiva Junqueira

29/Apr/09 04:32

04/May/09 11:02

contrib-bookkeeper

0

ZOOKEEPER-380

We had initially assumed one ledger open at a time, and LedgerRecoverMonitor reads and closes a ledger without instantiating a QuorumEngine. Consequently, when closing a ledger, we have to shut down a QuorumEngine only if there is one. A poor way of implementing it is by having a for loop that iterates over all available QuorumEngines and shut them down. Given the assumptions of one ledger at a time and no QuorumEngine for LedgerRecoveryMonitor, this implementation works. However, as we move to a more flexible model in which multiple ledgers can be open and closed at arbitrary times, the for loop choice no longer works.

The attached patch fixes this problem by just shutting down the QuorumEngine instance corresponding to the ledger being closed, and it checks if a QuorumEngine exists before submitting a STOP operation. The patch also includes a test that opens four ledger, and then sequentially writes to the each ledger and closes it.

214108

No Perforce job exists for this issue.

1

33102

29/Apr/09 6:24 AM

10 years, 47 weeks, 3 days ago

0|i0611z:

ZooKeeper

ZOOKEEPER-386

improve java cli shell

Improvement

Closed

Minor

Fixed

Henry Robinson

Patrick D. Hunt

27/Apr/09 18:43

08/Jul/09 16:24

07/May/09 18:35

3.2.0

java client

0

1

ZOOKEEPER-388

copied from the following comments by Vegard
https://issues.apache.org/jira/browse/ZOOKEEPER-364?focusedCommentId=12696002&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12696002
------------------------------------------------------------
Possible improvements:

Default host list to localhost:2181 when not given.

Don't print a ton of debug, it looks like both cli_mt and zkCli.sh does this. Make a simple ls that lists subnodes one line at a time and a simple cat that lists only the node content, simple put etc.

Shell with gnu readline capabilities and tab completion on node names would be a big improvement of usability during testing and debugging.

47853

No Perforce job exists for this issue.

5

33492

29/Apr/09 2:33 PM

10 years, 46 weeks, 6 days ago

add readline support via jline to the java client shell.

Reviewed

0|i063gn:

ZooKeeper

ZOOKEEPER-385

crctest failed on hudson patch test

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

27/Apr/09 18:38

08/Jul/09 16:24

22/May/09 03:43

3.2.0

server, tests

0

1

http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/53/testReport/org.apache.zookeeper.server/CRCTest/testChecksums/

crctest failed with

Error Message

Unreasonable length = 518291091

Stacktrace

java.io.IOException: Unreasonable length = 518291091
at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:101)
at org.apache.zookeeper.server.DataNode.deserialize(DataNode.java:116)
at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:109)
at org.apache.zookeeper.server.DataTree.deserialize(DataTree.java:954)
at org.apache.zookeeper.server.util.SerializeUtils.deserializeSnapshot(SerializeUtils.java:91)
at org.apache.zookeeper.server.persistence.FileSnap.deserialize(FileSnap.java:125)
at org.apache.zookeeper.server.CRCTest.testChecksums(CRCTest.java:146)

47854

No Perforce job exists for this issue.

1

33103

22/May/09 2:13 AM

10 years, 44 weeks, 6 days ago

Reviewed

0|i06127:

ZooKeeper

ZOOKEEPER-384

keeper exceptions missing path

Bug

Closed

Minor

Fixed

Patrick D. Hunt

27/Apr/09 16:55

08/Jul/09 16:24

05/May/09 00:55

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

java client

0

1

keeper exceptions thrown by the java client don't include path, having path helps in debugging.

47855

No Perforce job exists for this issue.

1

33104

27/Apr/09 6:02 PM

10 years, 47 weeks, 2 days ago

Reviewed

0|i0612f:

ZooKeeper

ZOOKEEPER-383

Asynchronous version of createLedger()

New Feature

Closed

Major

Fixed

Flavio Paiva Junqueira

Utkarsh Srivastava

27/Apr/09 16:35

01/May/13 22:29

28/May/09 01:01

3.2.0

contrib-bookkeeper

0

1

ZOOKEEPER-400

While there are async versions for read and write, there is no async version for creating a ledger. This can cause applications to have to change their whole thread design.

It should be easier and more consistent to add an async version of createLedger().

47856

No Perforce job exists for this issue.

6

33493

05/May/09 7:09 AM

10 years, 43 weeks, 5 days ago

Asynchronous calls for create ledger with changes in callbacks using ledger handle rather than ledgerid.

Reviewed

0|i063gv:

ZooKeeper

ZOOKEEPER-382

zookeeper cpp tests fails on 64 bit machines with gcc 4.1.2

Bug

Closed

Major

Fixed

Mahadev Konar

27/Apr/09 14:30

08/Jul/09 16:24

29/Apr/09 15:12

3.2.0

0

1

the zookeeper c tests fail on 64 bit machines with gcc 4.1.2 with the following error

[exec] /workspace/zootestbuild/trunk/src/c/tests/TestClient.cc: In static member function 'static void Zookeeper_simpleSystem::statCompletion(int, const Stat*, const void*)':
[exec] /workspace/zootestbuild/trunk/src/c/tests/TestClient.cc:273: error: cast from 'const void*' to 'int' loses precision
[exec]/workspace/zootestbuild/trunk/src/c/tests/TestClient.cc: In static member function 'static void Zookeeper_simpleSystem::voidCompletion(int, const void*)':
[exec] /workspace/zootestbuild/trunk/src/c/tests/TestClient.cc:291: error: cast from 'const void*' to 'int' loses precision

47857

No Perforce job exists for this issue.

1

33105

29/Apr/09 2:00 AM

10 years, 48 weeks ago

Reviewed

0|i0612n:

ZooKeeper

ZOOKEEPER-381

The ZooKeeper front page tells us nothing about what ZooKeeper does

Improvement

Closed

Trivial

Fixed

Grant Ingersoll

24/Apr/09 13:07

08/Jul/09 16:24

24/Apr/09 13:55

3.2.0

documentation

0

Currently, http://hadoop.apache.org/zookeeper does not contain a description of what ZK is. The patch to follow copies the opening text from the Wiki and adds it to a section above the Getting Started section.

47858

No Perforce job exists for this issue.

1

33494

24/Apr/09 1:53 PM

10 years, 48 weeks, 6 days ago

update the zookeeper hadoop site frontpage to give out a description of zookeeper.

Reviewed

0|i063h3:

ZooKeeper

ZOOKEEPER-380

bookkeeper should have a streaming api so that its easier to store checpoints/snapshots in bookkeeper.

Improvement

Closed

Major

Fixed

Mahadev Konar

23/Apr/09 17:12

08/Jul/09 16:24

06/May/09 05:02

3.1.0, 3.1.1

3.2.0

contrib-bookkeeper

0

1

ZOOKEEPER-387, ZOOKEEPER-365

currently, bookkeeper api allows just a bytes interface which is
ld.write(bytes).

We should have an interface like

Stream s = ledger.createStream() (I am not very sure of the interface right now but will post a more concrete one after giving it a little more thought)

now this stream can be used to wirte checkpoints as
s.write(bytes)
and then closed to s.close() to close the snapshot.
This api could use the current api to implement snapshots as chunks of bytes (buffered by stream s) that can be written via ld.write(bytes).

47859

No Perforce job exists for this issue.

6

33495

30/Apr/09 9:25 AM

10 years, 46 weeks, 6 days ago

Incompatible change

0|i063hb:

ZooKeeper

ZOOKEEPER-379

cppunit tests failed during hudson patch tests

Bug

Closed

Major

Invalid

Mahadev Konar

Giridharan Kesavan

21/Apr/09 02:30

08/Jul/09 16:24

21/May/09 20:27

3.2.0

tests

0

1

patch test build failed with " equality assertion failed"
link: http://hudson.zones.apache.org/hudson/view/ZooKeeper/job/Zookeeper-Patch-vesta.apache.org/44/console

Mahadev,
Could you take a look?
Tnx!

214107

No Perforce job exists for this issue.

0

33106

21/Apr/09 12:36 PM

10 years, 45 weeks ago

0|i0612v:

ZooKeeper

ZOOKEEPER-378

contribution of Net::ZooKeeper Perl module

New Feature

Closed

Major

Fixed

Chris Darroch

20/Apr/09 13:36

08/Jul/09 16:24

27/Apr/09 14:09

3.2.0

0

As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe.

I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/.

47860

No Perforce job exists for this issue.

2

33496

20/Apr/09 1:43 PM

10 years, 48 weeks ago

Net::ZooKeeper Perl module included under src/contribs/zkperl.

Reviewed

0|i063hj:

ZooKeeper

ZOOKEEPER-377

running ant cppunit tests, a failure still results in BUILD SUCCESSFUL

Bug

Closed

Critical

Fixed

Giridharan Kesavan

Patrick D. Hunt

17/Apr/09 14:14

08/Jul/09 16:24

20/Apr/09 18:16

3.2.0

0

1

Giri, can you take a look at this?

I ran "ant test"

looks like this test failed:
[exec] Zookeeper_operations::testOperationsAndDisconnectConcurrently1 : assertion

then later...

[exec]
[exec] /home/phunt/dev/workspace/svnzk_apache/src/c/tests/TestOperations.cc:551: Assertion: equality assertion failed [Expected: -4, Actual : 0, ZCONNECTIONLOSS != rc]
[exec] Failures !!!
[exec] Run: 38 Failure total: 1 Failures: 1 Errors: 0
[exec] make: *** [run-check] Error 1
[exec] Result: 2

test-core:

test-contrib:

BUILD SUCCESSFUL
Total time: 15 minutes 39 seconds

47861

No Perforce job exists for this issue.

1

33107

20/Apr/09 12:36 AM

10 years, 49 weeks, 1 day ago

Reviewed

0|i06133:

ZooKeeper

ZOOKEEPER-376

ant test target re-compiles cppunit code every time

Bug

Closed

Minor

Fixed

Patrick D. Hunt

15/Apr/09 18:10

08/Jul/09 16:24

07/May/09 18:38

3.2.0

build, c client

0

1

Giri, can you take a look at this:

1) looks like autoreconf is always run, which means that a re-configure/re-make is run each time the tests are run using ant

2) tabs were introduced to the build when cppunit changes were made, please only use spaces (would be great if you could fix this too -- hard to read in my editor)

Thanks!

47862

No Perforce job exists for this issue.

1

33108

01/May/09 12:27 AM

10 years, 46 weeks, 6 days ago

Reviewed

0|i0613b:

ZooKeeper

ZOOKEEPER-375

zoo_add_auth only retains most recent auth on re-sync

Bug

Closed

Blocker

Fixed

Mahadev Konar

Patrick D. Hunt

15/Apr/09 12:03

08/Jul/09 16:24

09/Jun/09 15:01

3.1.0, 3.1.1

3.2.0

c client

0

1

zoo_add_auth doesn't maintain a list of auths - it only stores the most recent auth send to the server. As a result on re-sync to the cluster it will lose (not reregister) any auths prior to the most recent.

This code should maintain a list of auths similar to the java code. Be sure to free the memory in close.

47863

No Perforce job exists for this issue.

3

33109

26/May/09 11:51 PM

10 years, 42 weeks ago

Reviewed

0|i0613j:

ZooKeeper

ZOOKEEPER-374

Uninitialized struct variable in C causes warning which is treated as an error

Bug

Closed

Trivial

Fixed

Patrick D. Hunt

Nitay Joffe

10/Apr/09 05:28

08/Jul/09 16:24

16/Apr/09 16:28

3.1.1

3.2.0

c client

0

1

i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5490)

nitay-joffes-macbook-pro:c nitay$ pwd
/Users/nitay/code/zookeeper/src/c

nitay-joffes-macbook-pro:c nitay$ make
make all-am
/bin/sh ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -MT zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c -o zookeeper.lo `test -f 'src/zookeeper.c' || echo './'`src/zookeeper.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I./include -I./tests -I./generated -Wall -Werror -g -O2 -MT zookeeper.lo -MD -MP -MF .deps/zookeeper.Tpo -c src/zookeeper.c -fno-common -DPIC -o .libs/zookeeper.o
cc1: warnings being treated as errors
src/zookeeper.c: In function 'zoo_add_auth':
src/zookeeper.c:2378: warning: 'auth.buff' may be used uninitialized in this function
src/zookeeper.c:2378: warning: 'auth.len' may be used uninitialized in this function
make[1]: *** [zookeeper.lo] Error 1
make: *** [all] Error 2

Need to set auth.buff and auth.len to zero.

47864

No Perforce job exists for this issue.

1

33110

10/Apr/09 11:47 AM

10 years, 49 weeks, 6 days ago

Reviewed

0|i0613r:

ZooKeeper

ZOOKEEPER-373

One thread per bookie

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

09/Apr/09 11:41

08/Jul/09 16:24

17/Apr/09 17:29

3.1.1

3.2.0

contrib-bookkeeper

0

Currently, if a client is writing to multiple ledgers and these ledgers overlap on some bookie, there will be as many threads for such a bookie as the number of ledgers writing to it. Consequently, if a client writes to many ledgers simultaneously, it may end up with an undesirably large number of threads. I don't have a concrete proposal yet, but I suspect it is as simple as keeping an array of BookieHandle objects, one per bookie, and having each BookieHandle object shared by all ledgers.

47865

No Perforce job exists for this issue.

3

33497

15/Apr/09 2:33 PM

10 years, 49 weeks, 3 days ago

Reviewed

0|i063hr:

ZooKeeper

ZOOKEEPER-372

Use ZooKeeper server identifier with AuthFastLeaderElection

Improvement

Open

Minor

Unresolved

Unassigned

Flavio Paiva Junqueira

09/Apr/09 11:18

01/May/13 22:29

leaderElection

0

1

ZOOKEEPER-577

Currently, we use InetSocketAddress objects to identify peers in AuthFastLeaderElection. We should change it to be consistent with the TCP version and to enable us to use with the feature of ZOOKEEPER-29.

214106

No Perforce job exists for this issue.

0

68473

10 years, 51 weeks ago

0|i0c39j:

ZooKeeper

ZOOKEEPER-371

to implement jdiff

Improvement

Closed

Major

Fixed

Giridharan Kesavan

08/Apr/09 09:06

08/Jul/09 16:24

22/Apr/09 13:53

3.2.0

build

0

ZOOKEEPER-148

to implement jdiff

47866

No Perforce job exists for this issue.

2

33498

14/Apr/09 12:04 PM

10 years, 49 weeks ago

Reviewed

0|i063hz:

ZooKeeper

ZOOKEEPER-370

Fix critical problems reported by findbugs

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

07/Apr/09 17:26

08/Jul/09 16:24

08/Apr/09 17:10

3.1.1

3.2.0

leaderElection

0

1

There are a couple of problems pointed out by findbugs and that we can easily fix.

47867

No Perforce job exists for this issue.

2

33111

07/Apr/09 9:34 PM

10 years, 51 weeks ago

Reviewed

0|i0613z:

ZooKeeper

ZOOKEEPER-369

remove multiple configs in server/quorums.

Improvement

Open

Major

Unresolved

Unassigned

Mahadev Konar

06/Apr/09 18:17

14/Dec/19 06:06

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.7.0

0

2

ZOOKEEPER-279

currently we have mutliple configs serverconfig and quorumpeerconfig for server and quorums. We can throw away one of these and just keep one of them with some methods returning default values like -1 or null for options that are not available.

70798

No Perforce job exists for this issue.

0

68474

08/Apr/09 7:11 PM

10 years, 45 weeks, 2 days ago

0|i0c39r:

ZooKeeper

ZOOKEEPER-368

Observers: core functionality

New Feature

Closed

Major

Fixed

Henry Robinson

Flavio Paiva Junqueira

06/Apr/09 16:58

26/Mar/10 13:24

18/Nov/09 14:08

3.3.0

quorum

2

8

ZOOKEEPER-581, ZOOKEEPER-107

Edit (Henry Robinson/henryr) 12/11/09:

This JIRA specifically concerns the implementation of non-voting peers called Observers, their documentation and their tests.

Explicit goals are 1. not breaking any current ZK functionality, 2. enabling at least one deployment scenario involving Observers, 3. documentation describing how to use the feature and 4. tests validating the correct behaviour of 2.

Non goals of this JIRA are 1. performance optimizations specific to Observers, 2. compatibility with every feature of ZooKeeper (in particular all leader election protocols), which are both to be addressed in future JIRAs.

See http://wiki.apache.org/hadoop/ZooKeeper/Observers for more detail of use cases, proposed design and usage.

See http://wiki.apache.org/hadoop/ZooKeeper/Observers/ReviewGuide for a brief commentary on the current patch.

-------------

Currently, all servers of an ensemble participate actively in reaching agreement on the order of ZooKeeper transactions. That is, all followers receive proposals, acknowledge them, and receive commit messages from the leader. A leader issues commit messages once it receives acknowledgments from a quorum of followers. For cross-colo operation, it would be useful to have a third role: observer. Using Paxos terminology, observers are similar to learners. An observer does not participate actively in the agreement step of the atomic broadcast protocol. Instead, it only commits proposals that have been accepted by some quorum of followers.

One simple solution to implement observers is to have the leader forwarding commit messages not only to followers but also to observers, and have observers applying transactions according to the order followers agreed upon. In the current implementation of the protocol, however, commit messages do not carry their corresponding transaction payload because all servers different from the leader are followers and followers receive such a payload first through a proposal message. Just forwarding commit messages as they currently are to an observer consequently is not sufficient. We have a couple of options:

1- Include the transaction payload along in commit messages to observers;
2- Send proposals to observers as well.

Number 2 is simpler to implement because it doesn't require changing the protocol implementation, but it increases traffic slightly. The performance impact due to such an increase might be insignificant, though.

For scalability purposes, we may consider having followers also forwarding commit messages to observers. With this option, observers can connect to followers, and receive messages from followers. This choice is important to avoid increasing the load on the leader with the number of observers.

47868

No Perforce job exists for this issue.

14

33499

25/Jun/09 10:16 AM

10 years, 18 weeks, 6 days ago

Observers functionality in zookeeper. please read docs/zookeeperObservers.pdf/.html for more information.

Reviewed

0|i063i7:

ZooKeeper

ZOOKEEPER-367

RecoveryTest failure - "unreasonable length" IOException

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

03/Apr/09 20:29

08/Jul/09 16:24

17/Apr/09 14:12

3.2.0

server

0

1

ZOOKEEPER-340

ubuntu 8.10 intrpid ibex, jvm 1.6.0_10

during local testing I received the attached recoverytest failure

47869

No Perforce job exists for this issue.

8

33112

13/Apr/09 6:41 PM

10 years, 49 weeks, 3 days ago

Reviewed

0|i06147:

ZooKeeper

ZOOKEEPER-366

Session timeout detection can go wrong if the leader system time changes

Bug

Resolved

Major

Duplicate

Benjamin Reed

03/Apr/09 18:12

23/Apr/14 16:46

3.5.0

quorum, server

0

3

ZOOKEEPER-1366

the leader tracks session expirations by calculating when a session will timeout and then periodically checking to see what needs to be timed out based on the current time. this works great as long as the leaders clock progresses at a steady pace. the problem comes when there are big (session size) changes in clock, by ntp for example. if time gets adjusted forward, all the sessions could timeout immediately. if time goes backward sessions that should timeout may take a lot longer to actually expire.

this is really just a leader issue. the easiest way to deal with this is to have the leader relinquish leadership if it detects a big jump forward in time. when a new leader gets elected, it will recalculate timeouts of active sessions.

66960

No Perforce job exists for this issue.

1

33113

20/Aug/10 6:25 PM

5 years, 48 weeks, 1 day ago

0|i0614f:

ZooKeeper

ZOOKEEPER-365

javadoc is wrong for setLast in LedgerHandle

Bug

Closed

Minor

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

03/Apr/09 17:56

08/Jul/09 16:24

29/Apr/09 15:16

3.2.0

contrib-bookkeeper

0

1

ZOOKEEPER-380

Note: the javadoc is wrong here:

/**
* Returns the last entry identifier submitted and increments it.
* @return long
*/
long setLast(long last){

also would be great to have javadoc for the legerrecoverymonitor getNextHint method. I was reviewing this code and it would have been helpful to know what to expect of this method. (possible return values, etc...)

47870

No Perforce job exists for this issue.

1

33114

28/Apr/09 5:39 PM

10 years, 48 weeks ago

Reviewed

0|i0614n:

ZooKeeper

ZOOKEEPER-364

command line interface for zookeeper.

New Feature

Resolved

Major

Invalid

Patrick D. Hunt

Mahadev Konar

03/Apr/09 13:38

23/Apr/14 16:34

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.5.0

0

2

HADOOP-6541

currently we have a shell based interface for zookeeper (which again isnt well published). we should have a well published cli based interface for zookeeper.

70767

No Perforce job exists for this issue.

0

68475

03/Apr/09 1:46 PM

5 years, 48 weeks, 1 day ago

0|i0c39z:

ZooKeeper

ZOOKEEPER-363

NPE when recovering ledger with no hint

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Luca Telloli

03/Apr/09 07:08

08/Jul/09 16:24

06/Apr/09 22:43

3.1.1

3.2.0

contrib-bookkeeper

0

1

When recovering a ledger, LedgerRecoveryMonitor currently start from the entry preceding the hint. if the hint is zero, then it causes an access out of the bounds of the bookie array in QuorumEngine, leading to the mentioned NPE.

47871

No Perforce job exists for this issue.

2

33115

03/Apr/09 7:13 AM

10 years, 51 weeks, 2 days ago

Reviewed

0|i0614v:

ZooKeeper

ZOOKEEPER-362

Issues with FLENewEpochTest

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

03/Apr/09 05:04

08/Jul/09 16:24

03/Apr/09 18:12

3.1.1

3.2.0

0

1

ZOOKEEPER-325

I have been able to identify two reasons that cause FLENewEpochTest to fail:

1- There is a race condition that is triggered when two peers try to establish a connection to each other for leader election. Basically, if they start roughly at the same time, the server with highest id will try to open two connections. The two competing connections will lead to one notification message to be lost. This message happens to be critical for this two process scenario;
2- The code to shut down a peer is not working well with the unit tests. For this particular unit test, we need to be able to shut down a peer completely to check the situation the test tries to reproduce. However, it seems that in some runs timing causes the other peers to believe it is still alive, and end up electing it. This peer, however, eventually shuts down and leader election fails.

47872

No Perforce job exists for this issue.

2

33116

03/Apr/09 11:35 AM

10 years, 51 weeks, 5 days ago

Reviewed

0|i06153:

ZooKeeper

ZOOKEEPER-361

integrate cppunit testing as part of hudson patch process.

New Feature

Closed

Major

Fixed

Giridharan Kesavan

Mahadev Konar

02/Apr/09 17:30

08/Jul/09 16:24

14/Apr/09 00:58

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

we need to test the c tests as part of our hudson patch testing process.

47873

No Perforce job exists for this issue.

2

33500

03/Apr/09 2:25 AM

10 years, 50 weeks, 1 day ago

Reviewed

0|i063if:

ZooKeeper

ZOOKEEPER-360

WeakHashMap in Bookie.java causes NPE

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

02/Apr/09 12:32

08/Jul/09 16:24

03/Apr/09 17:43

3.1.1

3.2.0

contrib-bookkeeper

0

1

We need a strong reference to prevent a key in masterKeys on Bookie.java to be garbage collected.

47874

No Perforce job exists for this issue.

1

33117

02/Apr/09 12:49 PM

10 years, 51 weeks, 5 days ago

Reviewed

0|i0615b:

ZooKeeper

ZOOKEEPER-358

Throw exception when ledger does not exist

Improvement

Closed

Minor

Fixed

Flavio Paiva Junqueira

Luca Telloli

01/Apr/09 07:42

08/Jul/09 16:24

02/Jun/09 17:15

3.1.1

3.2.0

contrib-bookkeeper

0

Currently, openLedger() in the BookKeeper client returns null if the ledger ID does not exist on ZK. Maybe it would be better to throw a specific exception so it can be handled by the client side.

47875

No Perforce job exists for this issue.

2

33501

11/May/09 7:13 AM

10 years, 43 weeks, 1 day ago

Reviewed

0|i063in:

ZooKeeper

ZOOKEEPER-357

ZooKeeper cannot actually take care of a Zoo

Bug

Resolved

Critical

Invalid

Unassigned

Benjamin Reed

01/Apr/09 01:02

02/Apr/09 12:20

1

0

Last week I setup an unattended ZooKeeper cluster that I expected to be able to take care of things while I was on vacation. I used 7 dedicated servers with two disks. I had done extensive burn in testing of the servers, so I fully expected the system to work problem free for the entire week. Indeed, when I returned yesterday none of the 7 servers had any hardware or software problem the entire time nor had there been any network problems. On the other hand the Zoo was in complete chaos. Even though there were no errors in the ZooKeeper log most of the animals were near starvation, except for the lions who had gotten loose and eaten some of the pigs. I'll not even go into the stench from the dirty cages!

Either the documentation needs to more clearly explain how to configure the server properly in the Zoo environment, or it should clearly state that ZooKeeper cannot take care of the Zoo out of the box!

214104

No Perforce job exists for this issue.

0

33119

01/Apr/09 1:57 AM

11 years ago

0|i0615r:

ZooKeeper

ZOOKEEPER-356

Masking bookie failure during writes to a ledger

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

27/Mar/09 08:18

08/Jul/09 16:24

24/Jun/09 01:08

3.2.0

contrib-bookkeeper

0

The idea of this jira is to work out the changes necessary to make a client mask the failure of a bookie while writing to a ledger. I'm submitting a preliminary patch, but before I submit a final one, I need to have 288 committed.

47876

No Perforce job exists for this issue.

9

33502

05/Jun/09 7:48 PM

10 years, 40 weeks, 1 day ago

0|i063iv:

ZooKeeper

ZOOKEEPER-355

make validatePath non public in Zookeeper client api.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

27/Mar/09 01:09

08/Jul/09 16:24

09/Apr/09 17:03

3.1.0, 3.1.1

3.2.0

0

1

make validatePath non public in Zookeeper client api.

47877

No Perforce job exists for this issue.

2

33120

03/Apr/09 5:40 PM

10 years, 50 weeks, 5 days ago

Reviewed

0|i0615z:

ZooKeeper

ZOOKEEPER-354

ZOOKEEPER-349 to fix javadoc warning in the source files

Sub-task

Closed

Major

Fixed

Mahadev Konar

Giridharan Kesavan

26/Mar/09 19:52

08/Jul/09 16:24

27/Mar/09 01:25

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

documentation

0

[javadoc] trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java:597: warning - Tag @see:illegal character: "60" in "#create(String, byte[], List<ACL>, CreateMode)"
[javadoc]trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java:597: warning - Tag @see:illegal character: "62" in "#create(String, byte[], List<ACL>, CreateMode)"
[javadoc]trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java:597: warning - Tag @see: can't find create(String, byte[], List<ACL>, CreateMode) in org.apache.zookeeper.ZooKeeper
[javadoc]trunk/src/java/main/org/apache/zookeeper/ZooKeeper.java:499: warning - @param argument "flags" is not a parameter name.
[javadoc] Building index for all the packages and classes...
[javadoc] Building index for all classes...
[javadoc] Generating trunk/build/docs/api/stylesheet.css...
[javadoc] 24 warnings

47878

No Perforce job exists for this issue.

1

33503

26/Mar/09 9:29 PM

11 years, 6 days ago

Reviewed

0|i063j3:

ZooKeeper

ZOOKEEPER-353

ZOOKEEPER-349 javadoc warnings needs to be fixed

Sub-task

Closed

Major

Fixed

Giridharan Kesavan

26/Mar/09 19:17

08/Jul/09 16:24

26/Mar/09 21:01

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

[javadoc] /.../SOMETHING/trunk/src/java/generated/org/apache/zookeeper/server/quorum/QuorumPacket.java:46: cannot find symbol
[javadoc] symbol : class OutputArchive
[javadoc] location: class org.apache.zookeeper.server.quorum.QuorumPacket
[javadoc] public void serialize(OutputArchive a_, String tag) throws java.io.IOException {
[javadoc] ^
[javadoc] /.../SOMETHING/trunk/src/java/generated/org/apache/zookeeper/server/quorum/QuorumPacket.java:63: cannot find symbol
[javadoc] symbol : class InputArchive
[javadoc] location: class org.apache.zookeeper.server.quorum.QuorumPacket
[javadoc] public void deserialize(InputArchive a_, String tag) throws java.io.IOException {
[javadoc] ^
[javadoc] Building index for all the packages and classes...
[javadoc] Building index for all classes...
[javadoc] Generating /.../SOMETHING/trunk/build/docs/api/stylesheet.css...
[javadoc] 4 errors
[javadoc] 124 warnings

47879

No Perforce job exists for this issue.

2

33504

26/Mar/09 8:48 PM

11 years, 6 days ago

Reviewed

0|i063jb:

ZooKeeper

ZOOKEEPER-352

ZOOKEEPER-349 to add standard ant targets required by test-patch.sh script

Sub-task

Closed

Major

Fixed

Giridharan Kesavan

26/Mar/09 18:31

08/Jul/09 16:24

26/Mar/09 20:45

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

47880

No Perforce job exists for this issue.

1

33505

26/Mar/09 8:45 PM

11 years, 6 days ago

add standard test targets required by test-patch.

Reviewed

0|i063jj:

ZooKeeper

ZOOKEEPER-351

ZOOKEEPER-349 to run checkstyle

Sub-task

Closed

Major

Fixed

Giridharan Kesavan

26/Mar/09 18:01

08/Jul/09 16:24

26/Mar/09 20:31

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

47881

No Perforce job exists for this issue.

1

33506

26/Mar/09 8:31 PM

11 years, 6 days ago

run checkstyle

Reviewed

0|i063jr:

ZooKeeper

ZOOKEEPER-350

ZOOKEEPER-349 to run rats for releaseaudit

Sub-task

Closed

Major

Fixed

Giridharan Kesavan

26/Mar/09 16:50

08/Jul/09 16:24

26/Mar/09 20:39

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

47882

No Perforce job exists for this issue.

1

33507

26/Mar/09 8:39 PM

11 years, 6 days ago

run rats for release audits.

Reviewed

0|i063jz:

ZooKeeper

ZOOKEEPER-349

to automate patch testing

Improvement

Closed

Major

Fixed

Giridharan Kesavan

26/Mar/09 16:49

08/Jul/09 16:24

27/Mar/09 13:31

3.2.0

build

0

ZOOKEEPER-350, ZOOKEEPER-351, ZOOKEEPER-352, ZOOKEEPER-353, ZOOKEEPER-354

47883

No Perforce job exists for this issue.

0

33508

27/Mar/09 1:31 PM

11 years, 2 days ago

Reviewed

0|i063k7:

ZooKeeper

ZOOKEEPER-348

Creating node with path ending in "/" with sequence flag set

Bug

Closed

Minor

Fixed

Patrick D. Hunt

Jeff Terrace

26/Mar/09 15:11

08/Jul/09 16:24

09/Apr/09 17:04

3.1.0, 3.1.1

3.2.0

c client

0

1

In 3.0.1, I could create a sequence node like this:
/nodes/0000001
like this:
string path = "/nodes/";
string value = "data";
int rc = zoo_acreate(zh, path.c_str(), value.c_str(), value.length(), &ZOO_OPEN_ACL_UNSAFE, ZOO_EPHEMERAL | ZOO_SEQUENCE, &czoo_created, &where);

In 3.1.1, this fails with error -8 (ZBADARGUMENTS).

Adding something after the "/" in the path makes the code work fine:
string path = "/nodes/n";

I assume something is checking if the path ends in "/" but not checking the sequence flag.

47884

No Perforce job exists for this issue.

0

33121

26/Mar/09 7:34 PM

10 years, 51 weeks ago

Reviewed

0|i06167:

ZooKeeper

ZOOKEEPER-347

zkfuse uses non-standard String

Bug

Closed

Minor

Fixed

Patrick D. Hunt

Nitay Joffe

25/Mar/09 20:13

08/Jul/09 16:24

08/Apr/09 18:39

3.1.0

3.2.0

contrib-zkfuse

0

1

gcc (GCC) 4.3.2 20081105 (Red Hat 4.3.2-7)

I'm getting an error compiling zkfuse:

g++ -DHAVE_CONFIG_H -I. -I.. -I/home/nitay/code/zookeeper-git/src/contrib/zkfuse/../../c/include -I/home/nitay/code/zookeeper-git/src/contrib/zkfuse/../../c/generated -I../include -I/usr/include -D_FILE_OFFSET_BITS=64 -D_REENTRANT -g -O2 -MT zkfuse.o -MD -MP -MF .deps/zkfuse.Tpo -c -o zkfuse.o zkfuse.cc
zkfuse.cc: In function 'int main(int, char**)':
zkfuse.cc:4282: error: 'String' does not name a type
zkfuse.cc:4283: error: 'file' was not declared in this scope
make[2]: *** [zkfuse.o] Error 1
make[2]: Leaving directory `/home/nitay/code/zookeeper-git/src/contrib/zkfuse/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/nitay/code/zookeeper-git/src/contrib/zkfuse'
make: *** [all] Error 2

4279 /**
4280 * Initialize log4cxx
4281 */
4282 const String file("log4cxx.properties");
4283 PropertyConfigurator::configureAndWatch( file, 5000 );
4284 LOG_INFO(LOG, "Starting zkfuse");

String is not standard, we should change it to std::string.

47885

No Perforce job exists for this issue.

1

33122

25/Mar/09 8:16 PM

10 years, 51 weeks ago

Reviewed

0|i0616f:

ZooKeeper

ZOOKEEPER-346

remove the kill command fro mthe client port.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

23/Mar/09 19:34

08/Jul/09 16:24

17/Apr/09 16:48

3.0.0, 3.0.1, 3.1.0

3.2.0

0

1

we should just remove the kill command from the client port. Its a security risk (though we do not have much security right now) to be able to kill the server from a tcp port without any authentication...

47886

No Perforce job exists for this issue.

2

33123

23/Mar/09 7:41 PM

10 years, 49 weeks, 3 days ago

removed the kill command from the client port.

Incompatible change, Reviewed

0|i0616n:

ZooKeeper

ZOOKEEPER-345

the CLIs should allow addAuth to be invoked

Improvement

Closed

Major

Fixed

Henry Robinson

Patrick D. Hunt

23/Mar/09 19:28

08/Jul/09 16:24

20/May/09 14:08

3.2.0

c client, java client

0

Not sure about the c client but the java CLI ZooKeeperMain does not allow addAuth to be called, it should have a command for this.

47887

No Perforce job exists for this issue.

2

33509

18/May/09 7:11 PM

10 years, 44 weeks, 6 days ago

Reviewed

0|i063kf:

ZooKeeper

ZOOKEEPER-344

doIO in NioServerCnxn: Exception causing close of session : cause is "read error"

Bug

Closed

Major

Invalid

Patrick D. Hunt

bryan thompson

18/Mar/09 14:45

08/Jul/09 16:24

06/May/09 13:04

3.1.0

3.2.0

java client, server

0

1

jdk1.6.0_07
Linux blade2 2.6.27.7-134.fc10.x86_64 #1 SMP Mon Dec 1 22:21:35 EST 2008 x86_64 x86_64 x86_64 GNU/Linux

I have been having a problem with zookeeper 3.0.1 and now with 3.1.0 where I see a lot of expired sessions. I am using a 16 node cluster which is all on the same local network. There is a single zookeeper instance (these are benchmarking runs).
The problem appears to be correlated with either run time or system load.\

Personally I think that it is system load because I have session session expired events under a Windows platform running zookeeper and the application (i.e., everthing is local) when the application load suddenly spikes. To me this suggests that the client is not able to renew (ping) the zookeeper service in a timely manner and is expired. But the log messages below with the "read error" suggest that maybe there is something else going on?

Zookeeper Configuration
#Wed Mar 18 12:41:05 GMT-05:00 2009
clientPort=2181
dataDir=/var/bigdata/benchmark/zookeeper/1
syncLimit=2
dataLogDir=/var/bigdata/benchmark/zookeeper/1
tickTime=2000

Some representative log messages are below.

Client side messages (from our app)
ERROR [main-EventThread] com.bigdata.zookeeper.ZLockImpl$ZLockWatcher.process(ZLockImpl.java:400) 2009-03-18 13:35:40,335 - Session expired: WatchedEvent: Server state change. New state: Expired : zpath=/benchmark/jobs/com.bigdata.service.jini.benchmark.ThroughputMaster/test_1/client1160/locknode
ERROR [main-EventThread] com.bigdata.zookeeper.ZLockImpl$ZLockWatcher.process(ZLockImpl.java:400) 2009-03-18 13:35:40,335 - Session expired: WatchedEvent: Server state change. New state: Expired : zpath=/benchmark/jobs/com.bigdata.service.jini.benchmark.ThroughputMaster/test_1/client1356/locknode

Server side messages:
WARN [NIOServerCxn.Factory:2181] org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:417) 2009-03-18 13:06:57,252 - Exception causing close of session 0x1201aac14300022 due to java.io.IOException: Read error
WARN [NIOServerCxn.Factory:2181] org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:417) 2009-03-18 13:06:58,198 - Exception causing close of session 0x1201aac1430000f due to java.io.IOException: Read error

214103

No Perforce job exists for this issue.

0

33124

18/Mar/09 4:39 PM

10 years, 47 weeks, 1 day ago

Reviewed

0|i0616v:

ZooKeeper

ZOOKEEPER-343

add tests that specifically verify the zkmain and qpmain classes

Improvement

Closed

Major

Fixed

Patrick D. Hunt

18/Mar/09 12:17

08/Jul/09 16:24

06/Apr/09 18:21

3.2.0

tests

0

We are missing tests for these two main() routines.

Add tests that verify standalone and quorum (2 servers is probably enough) by starting and connecting a client.

Use on-disk configuration files to configure these.
(ie verify starting with actual config files)

47888

No Perforce job exists for this issue.

2

33510

21/Mar/09 12:17 AM

10 years, 51 weeks, 2 days ago

Reviewed

0|i063kn:

ZooKeeper

ZOOKEEPER-342

improve configuration code - remove static config and use java properties

Improvement

Closed

Major

Fixed

Patrick D. Hunt

18/Mar/09 12:16

08/Jul/09 16:24

06/Apr/09 22:40

3.2.0

server

0

The current server/quorum config classes are essentially global variables. Need to fix configuration parsing, remove use of essentially global vars (static) and also cleanup the code generally.

Add tests specific to configurtion parsing.

47889

No Perforce job exists for this issue.

0

33511

06/Apr/09 10:40 PM

10 years, 51 weeks, 2 days ago

Reviewed

0|i063kv:

ZooKeeper

ZOOKEEPER-341

regression in QuorumPeerMain, tickTime from config is lost, cannot start quorum

Bug

Closed

Blocker

Fixed

Patrick D. Hunt

18/Mar/09 12:11

08/Jul/09 16:24

18/Mar/09 12:38

3.1.1, 3.2.0

quorum, server

0

1

ZOOKEEPER 330/336 caused a regression in QuorumPeerMain -- cannot reliably start a cluster due to missing tickTime.

47890

No Perforce job exists for this issue.

1

33125

18/Mar/09 12:38 PM

11 years, 2 weeks ago

Reviewed

0|i06173:

ZooKeeper

ZOOKEEPER-340

binaryinputarchive throws out runtimeexceptions for unreasonable length datastructures.

Bug

Closed

Major

Fixed

Mahadev Konar

17/Mar/09 18:16

08/Jul/09 16:24

17/Apr/09 19:19

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

jute, server

0

1

ZOOKEEPER-367

binaryinputarchive throws out runtimeexceptions for unreasonable length datastructures. We should change that to be IOExceptions so that we can handle partial writes to logs,, machine powerdown better.

47891

No Perforce job exists for this issue.

1

33126

10 years, 49 weeks, 6 days ago

0|i0617b:

ZooKeeper

ZOOKEEPER-339

Prevent from running ZK nodes with the same node ID

Improvement

Open

Major

Unresolved

Unassigned

Mahadev Konar

17/Mar/09 18:09

14/Dec/19 06:08

3.0.0, 3.0.1, 3.1.0

3.7.0

server

0

1

ZK server should catch a misconfiguration of nodes that use the same ID in myid file.

70764

No Perforce job exists for this issue.

0

68476

19/May/09 6:13 PM

10 years, 45 weeks, 2 days ago

0|i0c3a7:

ZooKeeper

ZOOKEEPER-338

zk hosts should be resolved periodically for loadbalancing amongst zk servers.

New Feature

Open

Major

Unresolved

Unassigned

Mahadev Konar

17/Mar/09 18:01

16/Jan/17 01:38

3.0.0, 3.0.1, 3.1.0

c client, java client

7

11

ZOOKEEPER-836, ZOOKEEPER-2184, ZOOKEEPER-1355, ZOOKEEPER-328

The list of host names passed to ZK init method is resolved only once. Had a corresponding DNS entry been changed, it
would not be refreshed by the ZK library,effectively preventing from proper load balancing.

214102

No Perforce job exists for this issue.

0

68477

19/Mar/09 2:19 PM

5 years, 34 weeks, 2 days ago

0|i0c3af:

ZooKeeper

ZOOKEEPER-337

improve logging in leader election lookForLeader method when address resolution fails

Improvement

Closed

Major

Fixed

Patrick D. Hunt

17/Mar/09 13:11

26/Sep/13 18:34

16/Apr/09 19:00

3.2.0

quorum

0

leader election has the following code:

requestPacket.setSocketAddress(server.addr);
LOG.info("Server address: " + server.addr);

this should be switched to have the info log first, set sock addr second.

The reason for this is that if the setSocketAddress fails SUN is not printing the address used. As a result it's verify difficult to debug this issue.

If we log the server address first, then if the setSockAddress fails we'll see both the address of the server and the exception detail (right now we just see the exception detail which does not include the invlaid address in invalidAddressException).

47892

No Perforce job exists for this issue.

3

33512

08/Apr/09 2:25 AM

10 years, 49 weeks, 6 days ago

Reviewed

0|i063l3:

ZooKeeper

ZOOKEEPER-336

single bad client can cause server to stop accepting connections

Improvement

Closed

Critical

Fixed

Henry Robinson

Patrick D. Hunt

12/Mar/09 14:37

08/Jul/09 16:24

09/Jun/09 01:05

3.2.0

c client, java client, server

0

2

One user saw a case where a single mis-programmed client was overloading the server with connections - the client was creating a huge number of sessions to the server. This caused all of the fds on the server to become used.

Seems like we should have some way of limiting (configurable override) the maximum number of sessions from a single client (say 10 by default?) Also we should output warnings when this limit is exceeded (or attempt to exceed).

47893

No Perforce job exists for this issue.

6

33513

05/May/09 12:18 AM

10 years, 42 weeks ago

Reviewed

0|i063lb:

ZooKeeper

ZOOKEEPER-335

zookeeper servers should commit the new leader txn to their logs.

Bug

Closed

Blocker

Fixed

Benjamin Reed

Mahadev Konar

04/Mar/09 13:55

17/Nov/19 05:05

14/Jun/11 01:14

3.1.0

3.4.0

server

0

9

ZOOKEEPER-1081, ZOOKEEPER-1082

ZOOKEEPER-1278, ZOOKEEPER-790, ZOOKEEPER-3608

currently the zookeeper followers do not commit the new leader election. This will cause problems in a failure scenarios with a follower acking to the same leader txn id twice, which might be two different intermittent leaders and allowing them to propose two different txn's of the same zxid.

47894

No Perforce job exists for this issue.

9

33127

02/Jun/10 2:45 PM

17 weeks, 4 days ago

Reviewed

0|i0617j:

ZooKeeper

ZOOKEEPER-334

bookkeeper benchmark (testclient.java) has compiling errros.

Bug

Closed

Major

Fixed

Mahadev Konar

27/Feb/09 18:55

08/Jul/09 16:24

03/Mar/09 17:49

3.1.0

3.2.0

contrib-bookkeeper

0

1

bookkeeper benchmark (testclient.java) has compiling errors.

47895

No Perforce job exists for this issue.

3

33128

03/Mar/09 4:29 PM

11 years, 4 weeks, 1 day ago

Reviewed

0|i0617r:

ZooKeeper

ZOOKEEPER-333

helgrind thread issues identified in mt c client code

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

27/Feb/09 16:56

08/Jul/09 16:24

11/Mar/09 20:31

3.1.1, 3.2.0

c client

0

2

helgrind generated a number of issues, I pulled a bunch of them. Most are related to the test, some are really issues with the mt zk client code though:

valgrind --tool=helgrind --log-file=helgrind_mt.out ./zktest-mt

==31294== Thread #2: pthread_cond_{timed}wait called with un-held mutex
==31294== at 0x4027F8F: pthread_cond_wait@* (hg_intercepts.c:560)
==31294== by 0x404D881: pthread_cond_wait@GLIBC_2.0 (in /lib/tls/i686/cmov/libpthread-2.8.90.so)
==31294== by 0x4028037: pthread_cond_wait@* (hg_intercepts.c:574)
==31294== by 0x809EBB7: pthread_cond_wait (PthreadMocks.cc:54)
==31294== by 0x80ABCF6: notify_thread_ready (mt_adaptor.c:136)
==31294== by 0x80ABE90: do_io (mt_adaptor.c:277)

==31294== Possible data race during write of size 4 at 0x42E9A58
==31294== at 0x8050D83: terminateZookeeperThreads(_zhandle*) (ZKMocks.cc:518)
==31294== by 0x805543B: DeliverWatchersWrapper::call(_zhandle*, int, int, char const*, watcher_object_list**) (ZKMocks.cc:261)
==31294== by 0x80520F7: __wrap_deliverWatchers (ZKMocks.cc:220)
==31294== by 0x80A287B: process_completions (zookeeper.c:1393)
==31294== by 0x80ABDAA: do_completion (mt_adaptor.c:332)

==31294== Possible data race during write of size 4 at 0xBEFF5F30
==31294== at 0x80589AF: Zookeeper_watchers::ConnectionWatcher::~ConnectionWatcher() (TestWatchers.cc:54)
==31294== by 0x805D062: Zookeeper_watchers::testDefaultSessionWatcher1() (TestWatchers.cc:438)
==31294== by 0x805608C: CppUnit::TestCaller<Zookeeper_watchers>::runTest() (TestCaller.h:166)

==31294== Possible data race during write of size 4 at 0x42EB104
==31294== at 0x80A03EE: queue_completion (zookeeper.c:1776)
==31294== by 0x80A3A44: zookeeper_process (zookeeper.c:1598)
==31294== by 0x80AC00B: do_io (mt_adaptor.c:309)

==31294== Thread #29: pthread_cond_{timed}wait called with un-held mutex
==31294== at 0x4027F8F: pthread_cond_wait@* (hg_intercepts.c:560)
==31294== by 0x404D881: pthread_cond_wait@GLIBC_2.0 (in /lib/tls/i686/cmov/libpthread-2.8.90.so)
==31294== by 0x4028037: pthread_cond_wait@* (hg_intercepts.c:574)
==31294== by 0x809EBB7: pthread_cond_wait (PthreadMocks.cc:54)
==31294== by 0x80AB9B3: wait_sync_completion (mt_adaptor.c:82)
==31294== by 0x80A1E82: zoo_wget (zookeeper.c:2517)
==31294== by 0x80A1F13: zoo_get (zookeeper.c:2497)

47896

No Perforce job exists for this issue.

3

33129

03/Mar/09 1:31 PM

11 years, 2 weeks, 6 days ago

Reviewed

0|i0617z:

ZooKeeper

ZOOKEEPER-332

c client issues (memory leaks) reported by valgrind

Bug

Closed

Blocker

Invalid

Unassigned

Patrick D. Hunt

27/Feb/09 15:04

08/Jul/09 16:24

04/Mar/09 13:25

3.1.0

3.1.1, 3.2.0

c client

0

Attaching valgrind log files.

1) getpwuid_r doesn't seem like it's due to us
2) the rest seem to be valid

214101

No Perforce job exists for this issue.

2

33130

03/Mar/09 2:43 PM

11 years, 4 weeks, 1 day ago

0|i06187:

ZooKeeper

ZOOKEEPER-331

Create a privacy policy for the ZooKeeper website

Task

Open

Minor

Unresolved

Unassigned

Patrick D. Hunt

27/Feb/09 14:10

documentation

0

It would be great to collect analytics about the visitors to the website and to do so, we need to create a privacy policy that tells visitors what we will collect.

See HADOOP-5190

214100

No Perforce job exists for this issue.

0

68478

11 years, 4 weeks, 6 days ago

0|i0c3an:

ZooKeeper

ZOOKEEPER-330

zookeeper standalone server does not startup with just a port and datadir.

Bug

Closed

Blocker

Fixed

Mahadev Konar

24/Feb/09 20:30

08/Jul/09 16:24

27/Feb/09 14:05

3.1.1, 3.2.0

0

1

ZOOKEEPER-326 made a change to zookeeperservermain.java that broke the starting of zookeeperserver with just the port and datadir.

47897

No Perforce job exists for this issue.

2

33131

26/Feb/09 3:16 PM

11 years, 4 weeks, 5 days ago

Reviewed

0|i0618f:

ZooKeeper

ZOOKEEPER-329

document how to integrate 3rd party authentication into ZK server ACLs

Improvement

Closed

Minor

Fixed

Benjamin Reed

Patrick D. Hunt

23/Feb/09 18:30

06/Dec/10 14:08

23/Jun/09 15:04

3.2.0

documentation

0

1

ZOOKEEPER-938

the docs mention that zk supports pluggable auth schemes but doesn't detail the API/examples. We should add this to the docs.

47898

No Perforce job exists for this issue.

2

33514

18/Jun/09 6:49 PM

10 years, 40 weeks, 1 day ago

0|i063lj:

ZooKeeper

ZOOKEEPER-328

We need to test the code that uses DNS lookups that return multiple results to enumerate the zookeeper servers

Test

Open

Minor

Unresolved

Unassigned

Benjamin Reed

23/Feb/09 17:57

14/Dec/19 06:07

3.7.0

c client, java client

0

1

ZOOKEEPER-338

Our client code uses the list of addresses returned from DNS as separate servers, so a zookeeper installation can have a single host name resolve to the addresses of all the zookeeper servers. this allows the zookeepers to change without changing the clients' configuration. the code is there, but we do not have tests for that code.

70747

No Perforce job exists for this issue.

0

68479

05/Mar/10 6:23 PM

9 years, 17 weeks, 3 days ago

0|i0c3av:

ZooKeeper

ZOOKEEPER-327

document effects (latency) of storing large amounts of data in znodes

Improvement

Closed

Minor

Fixed

Benjamin Reed

Patrick D. Hunt

23/Feb/09 14:11

08/Jul/09 16:24

24/Jun/09 21:56

3.2.0

documentation

0

znodes have max 1mb of data storage. using zk to store large amounts of data can negatively impact latency as seen by clients - the server needs to serve the data. often it's a better idea to store a "token" (say a uri) in the znode pointing to a large data block stored elsewhere (filesystem? memcached, etc...) this offloads zk and reduces latency impact

we should have details of this in the forrest docs.

47899

No Perforce job exists for this issue.

1

33515

24/Jun/09 5:04 AM

10 years, 40 weeks ago

0|i063lr:

ZooKeeper

ZOOKEEPER-326

standalone server ignores tickTime configuration

Bug

Closed

Minor

Fixed

Chris Darroch

20/Feb/09 15:12

08/Jul/09 16:24

23/Feb/09 18:56

3.0.0, 3.0.1, 3.1.0

3.1.1, 3.2.0

server

0

1

When using the ZooKeeper server in standalone mode, it ignores the tickTime setting in the configuration file and uses the DEFAULT_TICK_TIME of 3000 coded into ZooKeeperServer.java.

47900

No Perforce job exists for this issue.

1

33132

23/Feb/09 4:30 PM

11 years, 5 weeks, 2 days ago

Reviewed

0|i0618n:

ZooKeeper

ZOOKEEPER-325

FLENewEpoch test fails.

Bug

Closed

Major

Fixed

Unassigned

Mahadev Konar

19/Feb/09 19:04

08/Jul/09 16:24

07/Apr/09 19:08

3.1.0

3.2.0

quorum

0

1

ZOOKEEPER-362, ZOOKEEPER-305

linux 2.6.9-5.ELsmp

FLENewEpochTest fails quite frequently on my machine.

47901

No Perforce job exists for this issue.

1

33133

20/Feb/09 2:21 AM

10 years, 51 weeks, 2 days ago

0|i0618v:

ZooKeeper

ZOOKEEPER-324

do not materialize strings in the server

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

19/Feb/09 17:54

01/Dec/10 16:19

server

0

1

ZOOKEEPER-849

We convert paths and authentication information to strings rather than byte[] even though we could work just as well with byte[] for our needs since we don't really interpret the strings. we are just doing basic pattern matching. the only really string manipulations we do with paths is to look for '/', but we could easily to that with byte[] since we use utf8 encoding for the strings. by not materializing the strings we save time doing the serializations and also space since most (almost all) of our strings are ASCII and thus just one byte.

we could probably get by without even changing the jute spec if we make the generated classes check for a flag to see whether strings should be treated as byte[] or String.

214099

No Perforce job exists for this issue.

0

68480

25/Feb/09 7:47 PM

9 years, 17 weeks, 1 day ago

0|i0c3b3:

ZooKeeper

ZOOKEEPER-323

Log file reuse

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

19/Feb/09 17:44

23/Jun/11 13:27

server

1

4

ZOOKEEPER-1107

in some benchmarks i noticed that you can get much higher throughput using preallocated files rather than doing the grow-by-padding approach that we use. we should recycle log files that are no longer needed. this should give us a performance boost. it will also make the purgelog utility unnecessary. we should have a policy that we only keep X snapshots and associated logs and reuse older log files (and probably delete the older snapshots as well since the snapshot would be useless without the corresponding log file.)

70157

No Perforce job exists for this issue.

0

68481

23/Jun/11 1:27 PM

8 years, 40 weeks ago

0|i0c3bb:

ZooKeeper

ZOOKEEPER-322

move examples out of docs into contrib/examples

Improvement

Open

Major

Unresolved

Unassigned

Mahadev Konar

18/Feb/09 21:24

14/Dec/19 06:06

3.7.0

documentation

0

move examples out of docs into contrib/examples

70774

No Perforce job exists for this issue.

0

68482

19/May/09 6:14 PM

10 years, 45 weeks, 2 days ago

0|i0c3bj:

ZooKeeper

ZOOKEEPER-321

optmize session tracking in zookeeper.

New Feature

Resolved

Major

Won't Fix

Mahadev Konar

18/Feb/09 21:23

05/Dec/11 00:28

07/Mar/10 23:36

3.2.2

c client, java client, server

0

sometimes a lot of zookeeper clients are read only. For such clients we do not need the session tracking in zookeeper. Getting rid of session tracking for such clients will help us sclae much better.

session

214098

No Perforce job exists for this issue.

0

33516

19/Feb/09 5:39 PM

10 years, 3 weeks, 3 days ago

0|i063lz:

ZooKeeper

ZOOKEEPER-320

call auth completion in free_completions()

Bug

Closed

Major

Fixed

Chris Darroch

17/Feb/09 17:48

08/Jul/09 16:24

27/Feb/09 15:05

3.0.0, 3.0.1, 3.1.0

3.1.1, 3.2.0

c client

0

1

If a client calls zoo_add_auth() with an invalid scheme (e.g., "foo") the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code.

If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg().

In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session.

So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API.

Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319.

47902

No Perforce job exists for this issue.

3

33134

25/Feb/09 7:22 PM

11 years, 4 weeks, 5 days ago

Reviewed

0|i06193:

ZooKeeper

ZOOKEEPER-319

add locking around auth info in zhandle_t

Bug

Closed

Major

Fixed

Chris Darroch

17/Feb/09 17:32

08/Jul/09 16:24

27/Feb/09 14:49

3.0.0, 3.0.1, 3.1.0

3.1.1, 3.2.0

c client

0

1

Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their "main" thread. This function alters the elements of the auth_info structure in the zhandle_t structure.

Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock.

47903

No Perforce job exists for this issue.

3

33135

25/Feb/09 7:20 PM

11 years, 4 weeks, 5 days ago

Reviewed

0|i0619b:

ZooKeeper

ZOOKEEPER-318

remove locking in zk_hashtable.c or add locking in collect_keys()

Bug

Closed

Major

Fixed

Chris Darroch

17/Feb/09 15:52

08/Jul/09 16:24

05/Mar/09 19:45

3.0.0, 3.0.1, 3.1.0

3.2.0

c client

0

1

From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated.

If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()).

However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a "delivery" list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process().

I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events().

Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see.

47904

No Perforce job exists for this issue.

1

33136

25/Feb/09 4:05 PM

11 years, 3 weeks, 1 day ago

Marks some hashtable-related functions static and removes them from the public C API.

Reviewed

0|i0619j:

ZooKeeper

ZOOKEEPER-317

avoid exporting functions which may be static

Improvement

Open

Minor

Unresolved

Unassigned

Chris Darroch

17/Feb/09 14:16

17/Feb/09 14:35

3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.2.0

4.0.0

c client

0

A number of functions in the 3.x.x C API such as add_to_list() and getImpl() from zk_hashtable.c and data_result_checker() and child_result_checker() from zookeeper.c could be marked static and removed from the set of exported functions.

Since this change would remove functions from the API, it should wait for 4.0.0.

214097

No Perforce job exists for this issue.

0

68483

17/Feb/09 2:30 PM

11 years, 6 weeks, 2 days ago

0|i0c3br:

ZooKeeper

ZOOKEEPER-316

configure option --without-cppunit does not work

Bug

Open

Major

Unresolved

Unassigned

Mahadev Konar

17/Feb/09 14:02

14/Jul/09 14:14

3.1.0

c client, tests

1

0

configure option --without-cppunit does not work.

214096

No Perforce job exists for this issue.

0

33137

16/Apr/09 1:08 PM

10 years, 37 weeks, 2 days ago

0|i0619r:

ZooKeeper

ZOOKEEPER-315

add forrest docs for bookkeeper.

Improvement

Closed

Blocker

Fixed

Flavio Paiva Junqueira

Mahadev Konar

17/Feb/09 13:32

08/Jul/09 16:23

23/Jun/09 14:43

3.1.0

3.2.0

contrib-bookkeeper

0

we should have forrest docs for bookkeeper for
- how to install bookkeeper
- usage model
- programming examples for users
- FAQ

47905

No Perforce job exists for this issue.

3

33517

23/Apr/09 5:40 PM

10 years, 40 weeks, 1 day ago

0|i063m7:

ZooKeeper

ZOOKEEPER-314

add wiki docs for bookeeper.

Improvement

Closed

Major

Fixed

Benjamin Reed

Mahadev Konar

17/Feb/09 13:30

08/Jul/09 16:23

24/Jun/09 03:34

3.1.0

3.2.0

contrib-bookkeeper

0

we should have a wiki page for bookeeper for users to take a cursory look at what it is.

47906

No Perforce job exists for this issue.

0

33518

24/Jun/09 3:34 AM

10 years, 40 weeks, 1 day ago

0|i063mf:

ZooKeeper

ZOOKEEPER-313

Problem with successive leader failures when no client is connected

Bug

Resolved

Major

Duplicate

Mahadev Konar

Sunanda Bera

13/Feb/09 17:20

04/Mar/09 14:01

3.0.0, 3.0.1

3.1.1

server

0

all

Steps to reproduce:

Create a 3 node cluster . Run some transactions and then stop all clients. Make sure no other clients connect for the duration of the test.

Let L1 be the current leader. Bring down L1. Let L2 be the leader chosen. Let the third node be N3. Note that this will increase the txn id for N3's snapshot without any transaction being logged. Now bring up L1 -- same will happen for L1. Now bring down L2.

Both N3 and L1 now have snapshots with a transaction id greater than the last logged transaction. Whoever is elected leader will try to restore its state from the filesystem and fail.

One easy workaround is obviously to change the FileTxnSnapLog not to save a snapshot if zxid > last logged zxid. The correct solution is possibly to log a transaction for leader election as well.

214095

No Perforce job exists for this issue.

0

33138

13/Feb/09 5:49 PM

11 years, 4 weeks, 1 day ago

0|i0619z:

ZooKeeper

ZOOKEEPER-312

AUTH_FAILED state is unused

Improvement

Resolved

Major

Information Provided

Unassigned

Thomas White

13/Feb/09 15:52

12/Feb/20 10:06

0

1

Either the AUTH_FAILED state should be removed, or an AuthFailedException should cause the ZooKeeper client to transition to the AUTH_FAILED state.

70730

No Perforce job exists for this issue.

0

68484

19/May/09 6:15 PM

5 weeks, 1 day ago

0|i0c3bz:

ZooKeeper

ZOOKEEPER-311

handle small path lengths in zoo_create()

Improvement

Closed

Minor

Fixed

Chris Darroch

12/Feb/09 15:03

05/Sep/09 18:36

06/Aug/09 14:17

3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.2.0

3.2.1, 3.3.0

c client

0

1

The synchronous completion for zoo_create() contains the following code:\\

{noformat}
if (sc->u.str.str_len > strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc->u.str.str_len-1;
}
if (len > 0) {
memcpy(sc->u.str.str, res.path, len);
sc->u.str.str[len] = '\0';
}
{noformat}

In the case where the max_realpath_len argument to zoo_create() is 0, none of this code executes, which is OK. In the case where max_realpath_len is 1, a user might expect their buffer to be filled with a null terminator, but again, nothing will happen (even if strlen(res.path) is 0, which is unlikely since new node's will have paths longer than "/").

The name of the argument to zoo_create() is also a little misleading, as is its description ("the maximum length of real path you would want") in zookeeper.h, and the example usage in the Programmer's Guide:

{noformat}
int rc = zoo_create(zh,"/xyz","value", 5, &CREATE_ONLY, ZOO_EPHEMERAL, buffer, sizeof(buffer)-1);
{noformat}

In fact this value should be the actual length of the buffer, including space for the null terminator. If the user supplies a max_realpath_len of 10 and a buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the returned value to 9 bytes and put the null terminator in the second-last byte, leaving the final byte of the buffer unused.

It would be better, I think, to rename the realpath and max_realpath_len arguments to something like path_buffer and path_buffer_len, akin to zoo_set(). The path_buffer_len would be treated as the full length of the buffer (as the code does now, in fact, but the docs suggest otherwise).

The code in the synchronous completion could then be changed as per the attached patch.

Since this would change, slightly, the behaviour or "contract" of the API, I would be inclined to suggest waiting until 4.0.0 to implement this change.

47907

No Perforce job exists for this issue.

2

33519

10/Apr/09 3:55 PM

10 years, 34 weeks ago

The zoo_create() function is revised to take path_buffer and path_buffer_len arguments. The path_buffer will be filled with as much of the actual path of the new node fits into the buffer, including a null terminator. Unlike previous behavior, if path_buffer_len (formerly max_realpath_len) is 1, the buffer will be filled with a single null terminator.

Incompatible change

0|i063mn:

ZooKeeper

ZOOKEEPER-310

Coverity report on issues in C client code

Bug

Open

Minor

Unresolved

Mahadev Konar

Patrick D. Hunt

11/Feb/09 13:16

14/Dec/19 06:06

3.1.0

3.7.0

c client

0

Coverity found the following issues in the c code thatwe should look at/resolve:

1) zookeeper.c

Event unterminated_case: This case (value 0) is not terminated by a 'break' statement.
717 case 0:
718 errno = EHOSTDOWN;
Event fallthrough: The above case falls through to this one.
719 case -1:

Event unterminated_case: This case (value 0) is not terminated by a 'break' statement.
739 case 0:
740 errno = EHOSTDOWN;
Event fallthrough: The above case falls through to this one.
741 case -1:

Event negative_return_fn: Called negative-returning function "socket(2, 1, 0)"
Event var_assign: NEGATIVE return value of "socket" assigned to signed variable "zh->fd"
1099 zh->fd = socket(PF_INET, SOCK_STREAM, 0);
Event negative_returns: Tracked variable "zh->fd" was passed to a negative sink.
1100 setsockopt(zh->fd, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(int));

Event deref_ptr: Directly dereferenced pointer "cptr->buffer"
1308 cptr->buffer->curr_offset = get_buffer_len(oa);
Event check_after_deref: Pointer "cptr->buffer" dereferenced before NULL check
1309 if (!cptr->buffer) {

cli.c

Event returned_null: Function "strchr" returned NULL value (checked 4 out of 5 times)
Event var_assigned: Variable "ptr" assigned to NULL return value from "strchr"
532 char *ptr = strchr(buffer, '\n');
Event dereference: Dereferencing NULL value "ptr"

recordio.c

Event alloc_fn: Called allocation function "malloc"
Event var_assign: Assigned variable "buff" to storage returned from "malloc(12U)"
284 struct buff_struct *buff = malloc(sizeof(struct buff_struct));
Event leaked_storage: Variable "buff" goes out of scope
At conditional (1): "!(ia != NULL)" taking true path
285 if (!ia) return 0;

Event alloc_fn: Called allocation function "malloc"
Event var_assign: Assigned variable "buff" to storage returned from "malloc(12U)"
301 struct buff_struct *buff = malloc(sizeof(struct buff_struct));
Event leaked_storage: Variable "buff" goes out of scope
At conditional (1): "!(oa != NULL)" taking true path
302 if (!oa) return 0;

70797

No Perforce job exists for this issue.

0

33139

17/Jun/09 6:47 PM

10 years, 3 weeks, 3 days ago

0|i061a7:

ZooKeeper

ZOOKEEPER-309

core dump using zoo_get_acl()

Bug

Closed

Major

Fixed

Mahadev Konar

Chris Darroch

10/Feb/09 17:21

08/Jul/09 16:23

11/Mar/09 20:49

3.0.0, 3.0.1, 3.1.0

3.1.1, 3.2.0

c client

0

1

The zookeeper_process() function incorrectly calls the c.acl_result member of the completion_list_t structure when handling the completion from a synchronous zoo_get_acl() request. The c.acl_result member is set to SYNCHRONOUS_MARKER, which is a null pointer.

The attached patch removes this call.

47908

No Perforce job exists for this issue.

2

33140

10/Feb/09 5:57 PM

11 years, 2 weeks, 6 days ago

Reviewed

0|i061af:

ZooKeeper

ZOOKEEPER-308

improve the atomic broadcast performance 3x

Improvement

Closed

Major

Fixed

Benjamin Reed

10/Feb/09 01:43

12/Sep/09 17:00

19/Feb/09 19:24

3.2.0

server

0

1

We need a benchmark for zab, the atomic broadcast protocol of zookeeper. we can also do some small tweaks to make zab go much faster.

47909

No Perforce job exists for this issue.

4

33520

18/Feb/09 2:48 PM

10 years, 28 weeks, 5 days ago

Reviewed

0|i063mv:

ZooKeeper

ZOOKEEPER-307

Log exception stack traces

Improvement

Closed

Major

Invalid

Unassigned

Flavio Paiva Junqueira

09/Feb/09 13:50

08/Jul/09 16:23

19/May/09 18:16

3.1.0

3.2.0

0

For some exceptions, both on ZooKeeper and BookKeeper, we are calling printStackTrace explicitly instead of making a call to the logger.

214094

No Perforce job exists for this issue.

0

33521

19/May/09 6:16 PM

10 years, 45 weeks, 2 days ago

0|i063n3:

ZooKeeper

ZOOKEEPER-306

Add BookKeeper documentation

Improvement

Closed

Major

Duplicate

Unassigned

Flavio Paiva Junqueira

09/Feb/09 11:59

08/Jul/09 16:23

28/Apr/09 12:58

3.1.0

3.2.0

contrib-bookkeeper

0

1

Add detailed documentation about BookKeeper.

214093

No Perforce job exists for this issue.

0

33522

10 years, 48 weeks, 2 days ago

0|i063nb:

ZooKeeper

ZOOKEEPER-305

Replace timers with semaphores in FLENewEpochTest

Improvement

Closed

Minor

Fixed

Flavio Paiva Junqueira

07/Feb/09 09:08

08/Jul/09 16:23

01/Apr/09 20:36

3.1.0

3.2.0

tests

0

ZOOKEEPER-325

This patch replaces timers in the test with semaphores.

47910

No Perforce job exists for this issue.

2

33523

01/Apr/09 6:04 PM

10 years, 51 weeks, 5 days ago

Reviewed

0|i063nj:

ZooKeeper

ZOOKEEPER-304

factor out common methods from zookeeper.java

Improvement

Resolved

Major

Invalid

Thomas Koch

Mahadev Konar

06/Feb/09 17:19

23/Apr/14 17:54

3.1.0

3.5.0

java client, server

0

2

we need to factor out common methods from zookeeper.java to a commons directory for zookeeper so that it can be used both in client and server without each of them depending on each other.

70757

No Perforce job exists for this issue.

0

68485

19/May/09 6:18 PM

5 years, 48 weeks, 1 day ago

0|i0c3c7:

ZooKeeper

ZOOKEEPER-303

Bin scripts don't work on Mac

Bug

Closed

Major

Fixed

Thomas White

06/Feb/09 09:06

05/Jan/11 22:45

10/Feb/09 13:41

3.1.0

3.2.0

scripts

0

2

ZOOKEEPER-299, HADOOP-7089

Mac OS X

Macs don't support the -f option in readlink

{noformat}
$ bin/zkServer.sh start
JMX enabled by default
readlink: illegal option -- f
usage: readlink [-n] [file ...]
{noformat}

47911

No Perforce job exists for this issue.

1

33141

06/Feb/09 5:10 PM

11 years, 7 weeks, 1 day ago

Reviewed

0|i061an:

ZooKeeper

ZOOKEEPER-302

Quote values in JMX ObjectNames

Bug

Closed

Major

Fixed

Patrick D. Hunt

Thomas White

06/Feb/09 07:40

23/Mar/10 12:45

06/Feb/09 16:21

3.1.0

0

1

Some characters are not allowed in ObjectName values and need quoting, see http://java.sun.com/javase/6/docs/api/javax/management/ObjectName.html.

This came up with IPv6 addresses which contain a colon character.

47912

No Perforce job exists for this issue.

2

33142

06/Feb/09 2:24 PM

11 years, 7 weeks, 5 days ago

Reviewed

ipv6

0|i061av:

ZooKeeper

ZOOKEEPER-301

Util class to initialize Zookeeper with a set of user-specified bookies

New Feature

Closed

Minor

Fixed

Flavio Paiva Junqueira

Luca Telloli

06/Feb/09 04:57

26/Mar/10 13:30

07/Apr/09 07:40

contrib-bookkeeper

0

This patch provides a class named org.apache.bookkeeper.util.ZookeeperInitialization which provides basic initialization for bookies onto zookeeper reading from a text file. An example configuration file is provided in conf/zookeeper-initialize-bookies.conf

47913

No Perforce job exists for this issue.

1

33524

07/Apr/09 7:40 AM

10 years, 51 weeks, 2 days ago

0|i063nr:

ZooKeeper

ZOOKEEPER-300

zk jmx code is calling printStackTrace when creating bean name (should not be)

Bug

Closed

Major

Fixed

Patrick D. Hunt

05/Feb/09 18:24

08/Jul/09 16:23

14/May/09 15:16

3.1.0

3.2.0

jmx

0

1

remove printStackTrace from zk jmx code (review the rest of the code at the same time)

47914

No Perforce job exists for this issue.

1

33143

07/May/09 6:42 PM

10 years, 46 weeks ago

Reviewed

0|i061b3:

ZooKeeper

ZOOKEEPER-299

server startup scripts fail on a mac.

Bug

Closed

Major

Fixed

Unassigned

Mahadev Konar

05/Feb/09 18:22

08/Jul/09 16:23

12/Feb/09 14:05

3.2.0

0

1

ZOOKEEPER-303

JMX enabled by default
readlink: illegal option -- f
usage: readlink [-n] [file ...]

./zkEnv.sh no such file.

if you run bin/zkServer.sh start this is the error i get.
if I run it from the bin directory then it seems to be fine.

47915

No Perforce job exists for this issue.

0

33144

11 years, 7 weeks ago

0|i061bb:

ZooKeeper

ZOOKEEPER-298

some excecutables (scripts typ.) are not marked as such in tar generated by "ant tar"

Bug

Closed

Major

Invalid

Unassigned

Patrick D. Hunt

05/Feb/09 17:23

08/Jul/09 16:23

19/May/09 18:19

3.1.0

3.2.0

build

0

The configure scripts in src/c, the zkServer.sh in src/c/test, and some of the other scripts are not marked as executable when running "ant tar". The build.xml should be updated to mark appropriately.

214092

No Perforce job exists for this issue.

0

33145

10 years, 45 weeks, 2 days ago

0|i061bj:

ZooKeeper

ZOOKEEPER-297

centralize version numbering in the source/build

Improvement

Reopened

Minor

Unresolved

Diego de Oliveira

Patrick D. Hunt

05/Feb/09 17:21

05/Feb/20 07:17

3.5.0

3.7.0, 3.5.8

build

1

7

There are now three locations in the source repository that store the version number of the code:

1) build.xml
2) configure.ac
3) zookeeper_version.h

these all have to be managed when releasing/branching/etc... would be better if there were just one place - at the very least the configure.ac should be used to generate the version for zookeeper_version.h.

Can we do a better job (ie less) at this?

patch

60218

No Perforce job exists for this issue.

4

68486

19/Sep/11 6:38 AM

5 years, 50 weeks, 3 days ago

0|i0c3cf:

ZooKeeper

ZOOKEEPER-296

adopt APR-style versioning and compatibility guidelines for C API

Improvement

Open

Major

Unresolved

Unassigned

Chris Darroch

05/Feb/09 14:55

c client

0

Per a recent discussion on the ZooKeeper users mailing list regarding the API/ABI change introduced in 3.1.0 by ZOOKEEPER-255, I would suggest going forwards that the project adopt the versioning and compatibility guidelines of the Apache Portable Runtime (APR) project. These are well documented here:

http://apr.apache.org/versioning.html

I'd also suggest adopting the parallel installation procedure used by APR. This would mean that, for example, as of version 4.0.0 the ZooKeeper C library would be installed as libzookeeper_mt-4.so and the include files would be installed as zookeeper-4/zookeeper.h, etc.

The namespace cleanup I suggest in ZOOKEEPER-295 would fit well with such a change.

I should also point out the (rather mysterious) intent of the GNU libtool versioning system for libraries; while many projects seem to disregard it, it does have some value:

http://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html

214091

No Perforce job exists for this issue.

0

68487

11 years, 8 weeks ago

0|i0c3cn:

ZooKeeper

ZOOKEEPER-295

prevent multiple namespace pollution by C API

Improvement

Open

Major

Unresolved

Unassigned

Chris Darroch

05/Feb/09 14:45

03/Sep/10 13:21

3.0.0, 3.0.1, 3.1.0

4.0.0

c client

0

ZOOKEEPER-604

The ZOOKEEPER-6 issue touched on the problem of namespace pollution by the ZooKeeper C API; this report was closed but I don't think the problem has actually been substantially resolved.

There are multiple namespaces to consider. First, the names of the C include files should ideally have a common prefix, e.g., zoo_recordio.h, or else be concatenated into a single zookeeper.h file. The zookeeper.jute.h include file has a reasonably good name in this regard.

Second, all macros should ideally have a common prefix, e.g., ZOO_ or ZK_ or ZOOKEEPER_. Currently many exported constants (not macros) have the ZOO_ prefix, such as ZOO_PERM_READ, but error codes have a Z prefix, e.g., ZOK, ZNOTEMPTY.

Third, all functions should ideally have a common prefix, e.g., zoo_ or zk_ or zookeeper_. Many do already, but there is some variation, such as zookeeper_init(), zookeeper_process(), and there are also a large number of functions which have no prefix. These include many of the functions defined in recordio.h and zookeeper.jute.h, such as get_buffer() and serialize_Id(). Many others are simply used internally within the ZooKeeper C library and not declared in an external include file, but still pollute the caller's namespace, e.g., get_xid(), process_completions(), adaptor_init(), etc. All external symbols in the libraries should have a common prefix.

Fourth, all structure and type definitions should also have a common prefix, again, zoo_ or zk_ or zookeeper_. This is especially true of structures which currently have very generic names such as Id, Stat, and ACL from zookeeper.jute.h; buffer, iarchive, and oarchive from recordio.h; and clientid_t and watcher_fn from zookeeper.h. The zhandle_t structure should also be renamed to have the same prefix, e.g., zoo_handle_t.

The ZOOKEEPER-6 report includes the comment that the names in zookeeper.jute.h will be difficult to change because they affect the Java code and that there should be "limited exposure since jute naming starts with caps". It would be nice to think so, but I fear that a structure named Id or Stat is going to be pretty darn commonplace in other people's code. I would strongly recommend revising the entire set file, function, macro, type and structure names for 4.0.0.

214090

No Perforce job exists for this issue.

0

68488

05/Feb/09 2:50 PM

10 years, 13 weeks, 2 days ago

0|i0c3cv:

ZooKeeper

ZOOKEEPER-294

document or change memory allocations in C API completion handling

Improvement

Open

Major

Unresolved

Unassigned

Chris Darroch

05/Feb/09 14:08

3.0.0, 3.0.1, 3.1.0

c client

0

1

The C API zookeeper_process() function, which is used in the IO thread, generally copies data into memory locations provided by the caller (for example, when handling a string completion, it uses memcpy() to copy the string data into a user-provided buffer).

However, when handling string-list and ACL-list completions, memory is allocated by deserialize_GetChildrenResponse() and deserialize_GetACLResponse(), respectively, but not released afterward. Instead it is passed back to the user. This implies the user is responsible for calling deallocate_GetChildrenResponse() or deallocate_GetACLResponse(), respectively.

At a minimum, this needs to be extremely well-documented.

Perhaps better, the user should be able to pass an allocation function and a parameter to be passed to it. Then this function can called repeatedly as a callback in order to allocate the necessary memory. However, this does assume that the user's allocation function is MT-safe; that requirement would need to be documented if this approach is taken.

For example, when using the ZooKeeper C API with the Apache Portable Runtime, one would ideally be able to use APR memory pools for such allocation requests. To do that, the user could pass a wrapper function for apr_palloc() and a data parameter that included a private sub-pool. The sub-pool would be MT-safe so long as it is used exclusively by the IO thread.

So, something like zoo_get_children(..., zoo_alloc_func *alloc_func, void *alloc_data). This would obviously be a change most appropriate for a major version release, e.g., 4.0.0.

214089

No Perforce job exists for this issue.

0

68489

11 years, 8 weeks ago

0|i0c3d3:

ZooKeeper

ZOOKEEPER-293

zoo_set needs to be abi compatible (3.1 changed the signature), fix this by adding zoo_set2

Bug

Closed

Critical

Fixed

Patrick D. Hunt

05/Feb/09 13:43

13/Feb/09 16:18

05/Feb/09 16:19

3.1.0

c client

0

1

ZOOKEEPER-255 fixed an issue with zoo_set not providing access to stat structure, however this has broken b/w compatibility with previous releases.

We need to:

1) revert zoo_set to not have stat parameter (keep b/w compat)
2) add zoo_set2 method with stat param added to the function signature
3) add a version.h file to src/c/include that provides zoo version detail to clients

ie.
> #define ZOO_MAJOR_VERSION 3
> #define ZOO_MINOR_VERSION 1
> #define ZOO_PATCH_VERSION 0

(a new jira should be added to centralize version numbering, we now have the version number in 3 places in the source)

47916

No Perforce job exists for this issue.

1

33146

05/Feb/09 2:41 PM

11 years, 7 weeks, 6 days ago

Reviewed

0|i061br:

ZooKeeper

ZOOKEEPER-292

commit configure scripts (autotools) to svn for c projects and include in release

Improvement

Closed

Major

Fixed

Patrick D. Hunt

04/Feb/09 20:27

08/Jul/09 16:23

20/May/09 14:13

3.2.0

c client, contrib-zkfuse

0

ZOOKEEPER-281

We currently don't include configure scripts, just the .ac file itself. This means that users are required
to run "autoreconf; ./configure" rather than just configure.

We should commit the configure script(s) to svn (hadoop core does this btw) for our autotools based projects (currently c client api and zkfuse).

Also these should be included in the build.

47917

No Perforce job exists for this issue.

1

33525

05/Feb/09 5:48 AM

10 years, 44 weeks, 6 days ago

Reviewed

0|i063nz:

ZooKeeper

ZOOKEEPER-291

regression for legacy code using KeeperException.Code constants (due to 246)

Bug

Closed

Major

Fixed

Patrick D. Hunt

04/Feb/09 14:23

13/Feb/09 16:18

04/Feb/09 15:01

3.1.0

java client

0

1

ZOOKEEPER-246

In 246 the old constants were deprecated and replace with enum. In the process usage of the orig constants was broken for switch statements, cases require compiletime constants.

47918

No Perforce job exists for this issue.

1

33147

04/Feb/09 2:53 PM

11 years, 7 weeks, 6 days ago

Reviewed

0|i061bz:

ZooKeeper

ZOOKEEPER-289

add debug messages to nioserver select loop.

Improvement

Closed

Major

Fixed

Mahadev Konar

03/Feb/09 15:14

13/Feb/09 16:18

03/Feb/09 17:21

3.0.0, 3.0.1

3.1.0

0

add some debug messages for any unexpected behaviour on select i/o in NIOServerCnxn.

47919

No Perforce job exists for this issue.

2

33526

03/Feb/09 4:10 PM

11 years, 8 weeks, 1 day ago

Reviewed

0|i063o7:

ZooKeeper

ZOOKEEPER-288

Cleanup and fixes to BookKeeper

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

03/Feb/09 06:49

08/Jul/09 16:23

01/Apr/09 17:51

3.0.0

3.2.0

contrib-bookkeeper

0

We observed one race condition when multiple threads try to write concurrently. This patch should fix it. I will also remove some commented code.

47920

No Perforce job exists for this issue.

4

33527

25/Mar/09 1:22 PM

10 years, 51 weeks, 5 days ago

New security scheme: ledger and mac keys;
Optimized reads: client sends request to just a quorum;

Reviewed

0|i063of:

ZooKeeper

ZOOKEEPER-287

high cpu utilization caused by nioserver factory thread

Bug

Closed

Critical

Duplicate

Unassigned

Patrick D. Hunt

03/Feb/09 00:26

26/Mar/10 13:24

14/Oct/09 12:59

3.0.1

3.3.0

server

0

1

ZOOKEEPER-427

Linux 2.6.9-67.ELsmp #1 SMP Wed Nov 7 13:58:04 EST 2007 i686 i686 i386 GNU/Linux
the servers were running in vmware vms
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode)

a user reported that a long running server, part of a 2 server ensemble, started using 100%cpu (1 server of the ensemble, the other was fine).

mahadev tracked it down to a thread in the server running epoll in a tight loop - the thread was the nio server factory thread that selects on client fds.

72423

No Perforce job exists for this issue.

0

33148

10 years, 24 weeks, 1 day ago

Marking as 3.2 -- we should resolve or close this as unreproduceable.

0|i061c7:

ZooKeeper

ZOOKEEPER-286

Make GenerateLoad use InstanceContainers

Improvement

Closed

Major

Fixed

Benjamin Reed

02/Feb/09 11:20

13/Feb/09 16:18

02/Feb/09 17:44

3.1.0

0

ZOOKEEPER-285, ZOOKEEPER-284

GenerateLoad is a pain to setup and run. It would be nice if it used the same infrastructure as the system test so that we can more easily run performance tests.

47921

No Perforce job exists for this issue.

4

33528

02/Feb/09 1:16 PM

11 years, 8 weeks, 2 days ago

Reviewed

0|i063on:

ZooKeeper

ZOOKEEPER-285

the stat command does not return the correct mode

Bug

Closed

Major

Fixed

Benjamin Reed

02/Feb/09 06:56

13/Feb/09 16:18

02/Feb/09 17:46

3.0.0, 3.0.1

3.1.0

0

1

ZOOKEEPER-286

the stat command is always returning "standalone" it should return the actual mode it is in.

47922

No Perforce job exists for this issue.

0

33149

02/Feb/09 5:46 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i061cf:

ZooKeeper

ZOOKEEPER-284

JMX doesn't get the clientPort correctly

Bug

Closed

Minor

Fixed

Patrick D. Hunt

Benjamin Reed

02/Feb/09 06:55

13/Feb/09 16:18

02/Feb/09 17:51

3.1.0

0

1

ZOOKEEPER-286

The client port in JMX is always zero. it should be getting the client port from the cnxnfactory. it's a pretty easy fix. the really problem is that we have a setter for the clientPort as well, which is a bit more complicated to implement correctly. do we need a setter for the clientPort? i think we should make it a readonly attribute.

47923

No Perforce job exists for this issue.

0

33150

02/Feb/09 5:51 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i061cn:

ZooKeeper

ZOOKEEPER-283

Add more javadocs to BookKeeper

Improvement

Closed

Major

Duplicate

Unassigned

Flavio Paiva Junqueira

30/Jan/09 14:27

26/Mar/10 13:24

26/Jan/10 17:40

3.3.0

contrib-bookkeeper

0

1

Add more javadoc descriptors to BookKeeper code.

214088

No Perforce job exists for this issue.

0

33529

10 years, 9 weeks, 2 days ago

0|i063ov:

ZooKeeper

ZOOKEEPER-282

the getchildren api in zookeeper should return an iterator.

Improvement

Resolved

Major

Duplicate

Unassigned

Mahadev Konar

29/Jan/09 18:23

08/Dec/15 17:32

3.0.0, 3.0.1, 3.1.0

c client, java client, server

2

3

ZOOKEEPER-423, ZOOKEEPER-2260

214087

No Perforce job exists for this issue.

0

68490

08/Dec/15 5:32 PM

4 years, 15 weeks, 2 days ago

0|i0c3db:

ZooKeeper

ZOOKEEPER-281

autoreconf fails for /zookeeper-3.0.1/src/c/

Bug

Closed

Major

Fixed

Patrick D. Hunt

Maxim P. Dementiev

29/Jan/09 09:44

08/Jul/09 16:23

05/Mar/09 17:34

3.0.1

3.1.1, 3.2.0

c client

0

1

ZOOKEEPER-292

Linux dememax-laptop 2.6.27-gentoo-r8 #2 SMP Fri Jan 23 13:42:35 MSK 2009 i686 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz GenuineIntel GNU/Linux
autoconf (GNU Autoconf) 2.63
automake (GNU automake) 1.10.2
m4 (GNU M4) 1.4.11
aclocal (GNU automake) 1.10.2
ltmain.sh (GNU libtool) 1.5.26 (1.1220.2.493 2008/02/01 16:58:18)
basename (GNU coreutils) 6.10
gettext (GNU gettext-runtime) 0.17
GNU ld (GNU Binutils) 2.18

> autoreconf -i -f -v
autoreconf-2.63: Entering directory `.'
autoreconf-2.63: configure.ac: not using Gettext
autoreconf-2.63: running: aclocal --force
configure.ac:21: error: AC_SUBST: `DX_FLAG_[]DX_CURRENT_FEATURE' is not a valid shell variable name
acinclude.m4:77: DX_REQUIRE_PROG is expanded from...
acinclude.m4:117: DX_ARG_ABLE is expanded from...
acinclude.m4:178: DX_INIT_DOXYGEN is expanded from...
configure.ac:21: the top level
autom4te-2.63: /usr/bin/m4 failed with exit status: 1
aclocal-1.10: autom4te failed with exit status: 1
autoreconf-2.63: aclocal failed with exit status: 1
>

47924

No Perforce job exists for this issue.

4

33151

29/Jan/09 1:31 PM

11 years, 3 weeks, 1 day ago

Introduces compatibility with new GNU Autoconf version 2.63.

Fixed the m4 file but releases now include configure script so this should no longer be an issue generally for end users (an issue, but fixed, for developers)

Reviewed

0|i061cv:

ZooKeeper

ZOOKEEPER-279

Allow specialization of quorum config parsing (e.g. variable expansion in zoo.cfg)

Improvement

Closed

Minor

Fixed

Jean-Daniel Cryans

Nitay Joffe

26/Jan/09 14:16

24/May/13 23:11

05/Mar/09 16:34

3.1.0

3.1.1, 3.2.0

0

2

ZOOKEEPER-369

We would like to define certain parts of ZooKeeper's configuration using variables that get substituted. For example, we want the ZooKeeper quorum to be able to use a dataDir configured per user. In other words, something like:

tickTime=2000
dataDir=/tmp/zookeeper-${user.name}
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

I think Java already has a system for configuration that allows something like this using Properties?

47925

No Perforce job exists for this issue.

3

33530

27/Jan/09 3:58 PM

6 years, 43 weeks, 5 days ago

Reviewed

0|i063p3:

ZooKeeper

ZOOKEEPER-278

Create a test jar

Improvement

Resolved

Minor

Implemented

Unassigned

Nitay Joffe

26/Jan/09 14:10

11/Oct/13 12:33

tests

0

ZOOKEEPER-233

I am working on integrating ZooKeeper into HBase. I've found myself copying a lot of the test infrastructure code from places like zk.t.QuorumTest to create classes like MiniZooKeeper so that I can test my new additions in HBase. I think things would be a lot easier if we shipped a ZooKeeper test jar for others to use. Additionally, I think we should clean up the test code a bit so that the tests use some common infrastructure rather than each doing its own setup/teardown of a ZK cluster. I believe this is how things are done for testing Hadoop things HBase.

For more context, see hbase-1144.patch in https://issues.apache.org/jira/browse/HBASE-1144

I would do this myself but I can't currently contribute to Apache projects other than HBase because of company issues.

214086

No Perforce job exists for this issue.

0

68491

27/Jan/09 3:56 PM

6 years, 23 weeks, 6 days ago

0|i0c3dj:

ZooKeeper

ZOOKEEPER-277

Define PATH_SEPARATOR

Improvement

Open

Trivial

Unresolved

Unassigned

Nitay Joffe

23/Jan/09 19:29

14/Dec/19 06:08

3.7.0

c client, documentation, java client, server, tests

0

5

ZOOKEEPER-835

We should define a constant for PATH_SEPARATOR = "/" and use that throughout the code rather than the hardcoded "/". Users can be told to use this constant to be safe in case of future changes.

71223

No Perforce job exists for this issue.

0

68492

23/Jan/09 7:52 PM

8 years, 41 weeks, 1 day ago

0|i0c3dr:

ZooKeeper

ZOOKEEPER-276

Bookkeeper contribution

New Feature

Closed

Major

Fixed

Flavio Paiva Junqueira

Luca Telloli

20/Jan/09 06:54

13/Feb/09 16:18

30/Jan/09 14:30

3.1.0

0

7

BookKeeper is a system to reliably log streams of records. In BookKeeper, servers are "bookies", log streams are "ledgers", and each unit of a log (aka record) is a "ledger entry". BookKeeper is designed to be reliable; bookies, the servers that store ledgers can be byzantine, which means that some subset of the bookies can fail, corrupt data, discard data, but as long as there are enough correctly behaving servers the service as a whole behaves correctly; the meta data for BookKeeper is stored in ZooKeeper.

47926

No Perforce job exists for this issue.

7

33531

28/Jan/09 5:18 PM

11 years, 7 weeks, 4 days ago

Reviewed

0|i063pb:

ZooKeeper

ZOOKEEPER-275

Bug in FastLeaderElection

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

15/Jan/09 10:18

13/Feb/09 16:18

29/Jan/09 18:11

3.0.0, 3.0.1

3.1.0

leaderElection

0

1

I found an execution in which leader election does not make progress. Here is the problematic scenario:

- We have an ensemble of 3 servers, and we start only 2;
- We let them elect a leader, and then crash the one with lowest id, say S_1 (call the other S_2);
- We restart the crashed server.

Upon restarting S_1, S_2 has its logical clock more advanced, and S_1 has its logical clock set to 1. Once S_1 receives a notification from S_2, it notices that it is in the wrong round and it advances its logical clock to the same value as S_1. Now, the problem comes exactly in this point because in the current code S_1 resets its vote to its initial vote (its own id and zxid). Since S_2 has already notified S_1, it won't do it again, and we are stuck. The patch I'm submitting fixes this problem by setting the vote of S_1 to the one received if it satisfies the total order predicate ("received zxid" is higher or "received zxid is the same and received id is higher").

Related to this problem, I noticed that by trying to avoid unnecessary notification duplicates, there could be scenarios in which a server fails before electing a leader and restarts before leader election succeeds. This could happen, for example, when there isn't enough servers available and one available crashes and restarts. I fixed this problem in the attached patch by allowing a server to send a new batch of notifications if there is at least one outgoing queue of pending notifications empty. This is ok because we space out consecutive batches of notifications.

47927

No Perforce job exists for this issue.

4

33152

23/Jan/09 6:39 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i061d3:

ZooKeeper

ZOOKEEPER-274

Document options to connect Jconsole remotely

Improvement

Closed

Minor

Invalid

Patrick D. Hunt

Flavio Paiva Junqueira

15/Jan/09 09:58

08/Jul/09 16:23

21/May/09 17:46

3.2.0

jmx

0

Here is the command I used:

java -cp .:./zookeeper-dev.jar:/usr/local/apache-log4j-1.2.15/log4j-1.2.15.jar -Dlog4j.configuration=log4j_console.properties -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=12122 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false org.apache.zookeeper.server.quorum.QuorumPeerMain zoo.cfg

In addition to the ones in jira 234, we also need the following to connect remotely:
-Dcom.sun.management.jmxremote.port
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

This command assumes no authentication or secure connection through ssl. To use authentication, other options are necessary, such as the password file:
-Dcom.sun.management.jmxremote.password.file

In general, I found the following page a good source of information on this topic:

http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html

214085

No Perforce job exists for this issue.

0

33532

21/May/09 5:46 PM

10 years, 45 weeks ago

0|i063pj:

ZooKeeper

ZOOKEEPER-273

Zookeeper c client build should not depend on CPPUNIT

Bug

Closed

Major

Fixed

Patrick D. Hunt

Runping Qi

14/Jan/09 19:36

17/Feb/09 14:02

26/Jan/09 12:56

3.1.0

3.2.0

c client

0

1

One should be able to build Zookeeper C client libs on a machine without CPPUNIT installation.

A simple fix is to remove from configure.ac the following line:
M_PATH_CPPUNIT(1.10.2)

47928

No Perforce job exists for this issue.

3

33153

23/Jan/09 2:26 PM

11 years, 6 weeks, 2 days ago

Reviewed

0|i061db:

ZooKeeper

ZOOKEEPER-272

getChildren can fail for large numbers of children

Bug

Closed

Major

Fixed

Mahadev Konar

Joshua Tuberville

13/Jan/09 13:15

13/Feb/09 16:18

29/Jan/09 18:22

3.1.0

0

3

Zookeeper allows creation of an abritrary number of children, yet if the String array of children names exceeds 4,194,304 bytes a getChildren will fail because ClientCnxn$SendThread.readLength() throws an exception on line 490. Mahadev Konar questioned this byte limit's need. In any case consistency of create children, get children should exist.

47929

No Perforce job exists for this issue.

1

33154

13/Jan/09 1:53 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i061dj:

ZooKeeper

ZOOKEEPER-271

Better command line parsing in ZookeeperMain.

Improvement

Resolved

Minor

Fixed

Hartmut Lang

Mahadev Konar

08/Jan/09 23:00

01/May/13 22:29

18/Mar/12 02:19

3.0.0, 3.0.1

3.5.0

java client

0

1

ZOOKEEPER-1307, ZOOKEEPER-1408, ZOOKEEPER-1409

The command line parsing in zookeepermain is very basic.We should use some kind of cli parsing (commons-cli?) or something else that is standard and improve our command line parsing. This will remove the scattered code that we have in zookeepermain and we will have much better command line parsing.

70735

No Perforce job exists for this issue.

2

33533

19/May/09 6:21 PM

8 years, 1 week, 4 days ago

There are three incompatibilities introduced by this commit into the client shell:

1) now requires commons-cli
2) get no longer returns stat information by default, however there is a "-s" option that will result in the stat being included
3) a deprecated message is reported in some cases, when the old command format is used. As a result the output of the command may be different compared to client output prior to this change.

Incompatible change, Reviewed

0|i063pr:

ZooKeeper

ZOOKEEPER-270

remove NoSyncConnected in KeeperState - it's not used anywhere in the codebase.

Bug

Open

Major

Unresolved

Unassigned

Mahadev Konar

08/Jan/09 16:10

25/Jan/10 18:56

java client, server

0

ZOOKEEPER-40

ZOOKEEPER-265 makes some constants deprecated. We should remove the deprecated stuff int the next release.

214084

No Perforce job exists for this issue.

0

33155

08/Jan/09 4:53 PM

10 years, 9 weeks, 3 days ago

0|i061dr:

ZooKeeper

ZOOKEEPER-269

connectionloss - add more documentation to detail

Improvement

Closed

Minor

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

07/Jan/09 13:17

13/Feb/09 16:18

29/Jan/09 18:30

3.0.0, 3.0.1

3.1.0

documentation

0

ZOOKEEPER-264

discussion with user, this should be better documented:
--------------
There are basically 2 cases where you can see connectionloss:

1) you call an operation on a session that is no longer alive

2) you are disconnected from a server when there are pending async operations to that server (you made an async request which has not yet completed)

Patrick

Kevin Burton wrote:
> Can this be thrown when using multiple servers as long as > 1 of them is
> online?
> Trying to figure out of I should try some type of reconnect if a single
> machine fails instead of failing altogether.
>
> Kevin
>

47930

No Perforce job exists for this issue.

3

33534

26/Jan/09 1:27 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i063pz:

ZooKeeper

ZOOKEEPER-268

tostring on jute generated objects can cause NPE

Bug

Closed

Major

Fixed

Patrick D. Hunt

07/Jan/09 13:05

13/Feb/09 16:18

26/Jan/09 14:28

3.0.0, 3.0.1

3.1.0

java client, server

0

1

Jute still causing problems with tostring operations on generated code, need to review/cleanup the toCSV code

From user Kevin Burton:
---------------------------------------------
Creating this node with this ACL:
Created /foo
setAcl /foo world:anyone:w

Causes the exception included below.

It's an infinite loop so it's just called over and over again filling my
console.

I'm just doing an exists( path, true ); ... setting a watch still causes the
problem.

java.lang.NullPointerException
at org.apache.jute.Utils.toCSVBuffer(Utils.java:234)
at
org.apache.jute.CsvOutputArchive.writeBuffer(CsvOutputArchive.java:101)
at
org.apache.zookeeper.proto.GetDataResponse.toString(GetDataResponse.java:48)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at
org.apache.zookeeper.ClientCnxn$Packet.toString(ClientCnxn.java:230)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:586)
at
org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:626)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:852)
java.lang.NullPointerException
at org.apache.jute.Utils.toCSVBuffer(Utils.java:234)
at
org.apache.jute.CsvOutputArchive.writeBuffer(CsvOutputArchive.java:101)
at
org.apache.zookeeper.proto.GetDataResponse.toString(GetDataResponse.java:48)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at
org.apache.zookeeper.ClientCnxn$Packet.toString(ClientCnxn.java:230)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at
org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:586)
at
org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:626)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:852)

47931

No Perforce job exists for this issue.

1

33156

08/Jan/09 4:06 PM

11 years, 9 weeks, 2 days ago

Reviewed

0|i061dz:

ZooKeeper

ZOOKEEPER-267

java client incorrectly generating syncdisconnected event when in disconnected state

Bug

Closed

Major

Fixed

Patrick D. Hunt

05/Jan/09 17:56

13/Feb/09 16:18

27/Jan/09 19:24

3.0.0, 3.0.1

3.1.0

c client, java client

0

1

The java client (and someone should also review the c client) is generating a syncdisconnected even if the client is currently in the disconnected state. We saw this with a user running the java client against a down standalone server (server not running, connection refused) - the "syncdisconnected" is generated by the client lib each time a connection attempt (fails) is made. Should only be generated once.

47932

No Perforce job exists for this issue.

1

33157

06/Jan/09 12:11 PM

11 years, 9 weeks, 1 day ago

Reviewed

0|i061e7:

ZooKeeper

ZOOKEEPER-266

KeeperState missing javadoc for values

Bug

Closed

Major

Fixed

Patrick D. Hunt

05/Jan/09 15:40

13/Feb/09 16:18

08/Jan/09 16:22

3.0.0, 3.0.1

3.1.0

java client

0

1

KeeperState is missing documentation of the states. Should provide some basic details and refer users to the appropriate forrest doc for more detail (most likely prog guide).

47933

No Perforce job exists for this issue.

0

33158

08/Jan/09 4:22 PM

11 years, 12 weeks ago

0|i061ef:

ZooKeeper

ZOOKEEPER-265

remove (deprecate) unused NoSyncConnected from KeeperState

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

05/Jan/09 15:38

13/Feb/09 16:18

08/Jan/09 16:21

3.0.0, 3.0.1

3.1.0

c client, java client, server

0

deprecate NoSyncConnected in KeeperState - it's not used anywhere in the codebase.

what about "Unknown"? that seems to be unused as well, deprecate it as well?

47934

No Perforce job exists for this issue.

1

33535

08/Jan/09 4:07 PM

11 years, 11 weeks, 2 days ago

Reviewed

0|i063q7:

ZooKeeper

ZOOKEEPER-264

docs should include a state transition diagram for client state

Improvement

Closed

Minor

Fixed

Benjamin Reed

Patrick D. Hunt

05/Jan/09 15:00

08/Jul/09 16:23

25/Jun/09 18:43

3.0.0, 3.0.1

3.2.0

documentation

0

ZOOKEEPER-269

we should have a state transition diagram to help users understand client state transitions. perhaps the edges could indicate what might cause such a transition? (not sure if that will work). keep in mind for the states that the java/c clients have diff names for constants (not sure how to handle). This should be added to the programmer guide in the appropriate section.

47935

No Perforce job exists for this issue.

3

33536

13/Jan/09 9:09 AM

10 years, 39 weeks, 5 days ago

0|i063qf:

ZooKeeper

ZOOKEEPER-263

document connection host:port as comma separated list in forrest docs

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

05/Jan/09 14:50

13/Feb/09 16:18

27/Jan/09 19:22

3.0.0, 3.0.1

3.1.0

c client, documentation, java client

0

The java API docs talk about the host:port comma separated list, however the forrest docs don't mention this. it's confusing for users, they don't know that the connection string can be a list of host:port comma separated. We should mention in the programmer guide, perhaps even in the overview/started (mention in passing and reference the prog guide for full detail).

Also the java doc should be updated, right now we have a "see also" in the short constructor, to make it more explicit we should copy/paste the docs.

The c client should also be updated to detail that it's a comma separated list of host:port (similar to java). Right host we just say "host", need to be more explicit as to the format of the string.

47936

No Perforce job exists for this issue.

1

33537

27/Jan/09 7:22 PM

11 years, 9 weeks, 1 day ago

Reviewed

0|i063qn:

ZooKeeper

ZOOKEEPER-262

unnecesssarily complex reentrant zookeeper_close() logic

Improvement

Open

Minor

Unresolved

Unassigned

Chris Darroch

29/Dec/08 19:46

14/Dec/19 06:08

3.0.0, 3.0.1, 3.1.0, 3.2.0, 4.0.0

3.7.0

c client

0

While working on a wrapper for the C API I puzzled over the problem of how to determine when the multi-threaded adaptor's IO and completion threads had exited. Looking at the code in api_epilog() and adaptor_finish() it seemed clear that any thread could be the "last one out the door", and whichever was last would "turn out the lights" by calling zookeeper_close().

However, on further examination I found that in fact, the close_requested flag guards entry to zookeeper_close() in api_epilog(), and close_requested can only be set non-zero within zookeeper_close(). Thus, only the user's main thread can invoke zookeeper_close() and kick off the shutdown process. When that happens, zookeeper_close() then invokes adaptor_finish() and returns ZOK immediately afterward.

Since adaptor_finish() is only called in this one context, it means all the code in that function to check pthread_self() and call pthread_detach() if the current thread is the IO or completion thread is redundant. The adaptor_finish() function always signals and then waits to join with the IO and completion threads because it can only be called by the user's main thread.

After joining with the two internal threads, adaptor_finish() calls api_epilog(), which might seem like a trivial final action. However, this is actually where all the work gets done, because in this one case, api_epilog() sees a non-zero close_requested flag value and invokes zookeeper_close(). Note that zookeeper_close() is already on the stack; this is a re-entrant invocation.

This time around, zookeeper_close() skips the call to adaptor_finish() -- assuming the reference count has been properly decremented to zero! -- and does the actual final cleanup steps, including deallocating the zh structure. Fortunately, none of the callers on the stack (api_epilog(), adaptor_finish(), and the first zookeeper_close()) touches zh after this.

This all works OK, and in particular, the fact that I can be certain that the IO and completion threads have exited after zookeeper_close() returns is great. So too is the fact that those threads can't invoke zookeeper_close() without my knowing about it.

However, the actual mechanics of the shutdown seem unnecessarily complex. I'd be worried a bit about a new maintainer looking at adaptor_finish() and reasonably concluding that it can be called by any thread, including the IO and completion ones. Or thinking that the zh handle can still be used after that innocuous-looking call to adaptor_finish() in zookeeper_close() -- the one that actually causes all the work to be done and the handle to be deallocated!

I'll attach a patch which I think simplifies the code a bit and makes the shutdown mechanics a little more clear, and might prevent unintentional errors in the future.

68115

No Perforce job exists for this issue.

3

68493

02/Jan/09 8:16 PM

8 years, 37 weeks, 1 day ago

0|i0c3dz:

ZooKeeper

ZOOKEEPER-261

Reinitialized servers should not participate in leader election

Improvement

Resolved

Major

Fixed

Benjamin Reed

18/Dec/08 17:57

12/Jul/17 23:10

13/Jan/17 16:24

3.6.0

leaderElection, quorum

0

5

A server that has lost its data should not participate in leader election until it has resynced with a leader. Our leader election algorithm and NEW_LEADER commit assumes that the followers voting on a leader have not lost any of their data. We should have a flag in the data directory saying whether or not the data is preserved so that the the flag will be cleared if the data is ever cleared.

Here is the problematic scenario: you have have ensemble of machines A, B, and C. C is down. the last transaction seen by C is z. a transaction, z+1, is committed on A and B. Now there is a power outage. B's data gets reinitialized. when power comes back up, B and C comes up, but A does not. C will be elected leader and transaction z+1 is lost. (note, this can happen even if all three machines are up and C just responds quickly. in that case C would tell A to truncate z+1 from its log.) in theory we haven't violated our 2f+1 guarantee, since A is failed and B still hasn't recovered from failure, but it would be nice if when we don't have quorum that system stops working rather than works incorrectly if we lose quorum.

214083

No Perforce job exists for this issue.

0

68494

18/Dec/08 6:13 PM

3 years, 9 weeks, 5 days ago

0|i0c3e7:

ZooKeeper

ZOOKEEPER-260

document the recommended values for server id's

Improvement

Closed

Major

Fixed

Mahadev Konar

18/Dec/08 14:37

13/Mar/19 14:22

30/Jan/09 15:47

3.0.0, 3.0.1

3.1.0

0

ZOOKEEPER-2503

currently we do nto doucment the recommended values for server id's. we should document that. For some large values of server id's we might get negative session's and also might get duplicate sessions from two servers.

47937

No Perforce job exists for this issue.

2

33538

30/Jan/09 3:45 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i063qv:

ZooKeeper

ZOOKEEPER-259

cleanup the logging levels used (use the correct level) and messages generated

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

17/Dec/08 13:54

13/Feb/09 16:18

15/Jan/09 18:49

3.1.0

c client, java client, server

0

Cleanup logging:

make sure logging uses the correct level, esp error and warn

make sure the messages are meaningful (esp fix "fixmsg" logs)

47938

No Perforce job exists for this issue.

3

33539

15/Jan/09 6:44 PM

11 years, 10 weeks, 6 days ago

Reviewed

0|i063r3:

ZooKeeper

ZOOKEEPER-258

docs incorrectly state max client timeout as 60 seconds (it's based on server ticktime)

Bug

Closed

Minor

Fixed

Patrick D. Hunt

16/Dec/08 17:50

13/Feb/09 16:18

08/Jan/09 16:41

3.0.0, 3.0.1

3.1.0

documentation

0

1

The docs incorrectly state the max client timeout as 60 seconds.
http://hadoop.apache.org/zookeeper/docs/r3.0.1/zookeeperProgrammers.html#ch_zkSessions

the current server code has the following logic:
if (sessionTimeout < zk.tickTime * 2) {
sessionTimeout = zk.tickTime * 2;
}
if (sessionTimeout > zk.tickTime * 20) {
sessionTimeout = zk.tickTime * 20;
}

So really the docs should say max is 20*tickTime

47939

No Perforce job exists for this issue.

1

33159

08/Jan/09 4:41 PM

11 years, 11 weeks, 2 days ago

Reviewed

0|i061en:

ZooKeeper

ZOOKEEPER-257

Review logging level for WARN/ERROR log messages, some misclassified

Bug

Closed

Major

Duplicate

Patrick D. Hunt

15/Dec/08 19:49

13/Feb/09 16:18

06/Jan/09 19:22

3.0.0, 3.0.1

3.1.0

java client, server

0

Some of the WARN/ERROR log messages are incorrectly leveled (too high - many often should be INFO/DEBUG)

214082

No Perforce job exists for this issue.

0

33160

11 years, 12 weeks, 2 days ago

0|i061ev:

ZooKeeper

ZOOKEEPER-256

support use of JMX to manage log4j configuration at runtime

New Feature

Closed

Major

Fixed

Patrick D. Hunt

13/Dec/08 01:58

13/Feb/09 16:18

22/Dec/08 18:48

3.1.0

jmx

0

Log4j has code to support management at runtime using JMX. Need to add hooks to register the beans.

47940

No Perforce job exists for this issue.

1

33540

22/Dec/08 4:49 PM

11 years, 14 weeks, 2 days ago

Reviewed

0|i063rb:

ZooKeeper

ZOOKEEPER-255

zoo_set() api does not return stat datastructure.

Bug

Closed

Blocker

Fixed

Avery Ching

Mahadev Konar

12/Dec/08 18:18

13/Feb/09 16:18

22/Dec/08 16:40

3.0.0, 3.0.1

3.1.0

c client

0

1

the zoo_set() api does not return the stat datastructure. the java counterpart returns a stat with set api.

47941

No Perforce job exists for this issue.

2

33161

15/Dec/08 1:46 PM

11 years, 14 weeks, 2 days ago

Incompatible change, Reviewed

0|i061f3:

ZooKeeper

ZOOKEEPER-254

there is currently no way for a user to test session expiration in their code

Bug

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

12/Dec/08 16:44

13/Feb/09 16:18

22/Jan/09 17:39

3.1.0

tests

0

1

There's currently no way for a user to test session expiration in their code.

We don't have any unit/system tests that verify our code handles session expiration properly.

There should be a way to test session expiration.

I did notice that we have the ability to terminate a session using JMX mbean interface, however I'm not sure if this is useful in an automated testing context. Even if it is we should provide a wrapper for testing purposes - and add tests to our codebase which uses it.

47942

No Perforce job exists for this issue.

0

33162

11 years, 10 weeks ago

Reviewed

0|i061fb:

ZooKeeper

ZOOKEEPER-253

documentation of DataWatcher state transition is misleading regarding auto watch reset on reconnect

Bug

Closed

Minor

Fixed

Patrick D. Hunt

11/Dec/08 13:45

13/Feb/09 16:18

27/Jan/09 19:47

3.0.0, 3.0.1

3.1.0

documentation

1

2

the example code has

case SyncConnected:
// Everything is happy. Lets kick things off
// again by checking the existence of the znode
break;

this is misleading - it should indicate that the watches are automatically reset and therefor no call to exists is necessary

also fix this in the same doc (looks like its old detail, no longer valid), indicate that autoreset will happen on reconnect.

If the client-side ZooKeeper libraries can reestablish the communication channel to ZooKeeper, DataMonitor simply kicks everything off again with the call to ZooKeeper.exists(). If it gets an event for a znode, it calls ZooKeeper.exists() to find out what has changed.

47943

No Perforce job exists for this issue.

1

33163

27/Jan/09 7:47 PM

11 years, 9 weeks, 1 day ago

Reviewed

0|i061fj:

ZooKeeper

ZOOKEEPER-252

PurgeTxnLog is not handling the new dataDir directory structure

Bug

Closed

Critical

Fixed

Mahadev Konar

Patrick D. Hunt

10/Dec/08 18:06

13/Feb/09 16:18

30/Jan/09 20:19

3.0.0, 3.0.1

3.1.0

server

0

1

ZOOKEEPER-229

org.apache.zookeeper.server.PurgeTxnLog class has not been updated to handle the new directory structure imposed by the upgrade from v2 to v3 of ZooKeeper. In particular the dataDir now has a "version-2" subdirectory that stores all of the snaps/transactionallogs for version2 of the persistence layer.

I also note that the documentation of this class is particularly poor. I'm working on ZOOKEEPER-229 and would like to point to the API docs for this class regarding usage but they api docs are nonexistent

Also - I think it's important for the user to be able to specify the number of "backup" snaps and logs that should be kept -- right now it seems we delete all but the current snaps/txlogs. Either by count or by date -- ie "remove anything 5 days or older, with a minum of 3 most recents snaps (and accompanying txlogs)" seems like a pretty common user case (assuming the operator is doing system backups every X days, etc...)

in general this class needs some tlc - the formatting should also be cleaned up.

Also - the API docs for this and LogFormatter are not included in the build.xml "javadoc" target. These are user utilities so javadoc for these two classes should be included. I will fix this issue as part of ZOOKEEPER-229. I'm also updateing the forrest documention in 229 so don't worry about that either.

47944

No Perforce job exists for this issue.

4

33164

26/Jan/09 9:04 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i061fr:

ZooKeeper

ZOOKEEPER-251

NullPointerException stopping and starting Zookeeper servers

Bug

Closed

Blocker

Fixed

Mahadev Konar

Thomas Vinod Johnson

08/Dec/08 16:29

13/Feb/09 16:18

10/Dec/08 16:55

3.0.0, 3.0.1

3.1.0

server

0

2

Tested with JDK 1.5, Solaris, but I suspect it is not relevant in this case.

See the following thread for the original report:
http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200812.mbox/browser
Steps to reproduce:
1) Start a replicated zookeeper service consisting of 3 zookeeper (3.0.1) servers all running on the same host (of course, all using their own ports and log directories)
2) Create one znode in this ensemble (using the zookeeper client console, I issued 'create /node1 node1data').
3) Stop, then restart a single zookeeper server; moving onto the next one a few seconds later.
4) Go back to 3. After 4-5 iterations, the following should occur, with the failing server exiting:
java.lang.NullPointerException
at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:447)
at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:358)
at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.<init>(FileTxnLog.java:333)
at
org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:250)
at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:102)
at
org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:183)
at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:245)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:421)
2008-12-08 14:14:24,880 - INFO
[QuorumPeer:/0:0:0:0:0:0:0:0:2183:Leader@336] - Shutdown called
java.lang.Exception: shutdown Leader! reason: Forcing shutdown
at
org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:336)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:427)
Exception in thread "QuorumPeer:/0:0:0:0:0:0:0:0:2183"
java.lang.NullPointerException
at
org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:339)
at
org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:427)

The inputStream field is null, apparently because next is being called
at line 358 even after next returns false. Having very little knowledge
about the implementation, I don't know if the existence of hdr.getZxid()
>= zxid is supposed to be an invariant across all invocations of the
server; however the following change to FileTxnLog.java seems to make
the problem go away.
diff FileTxnLog.java /tmp/FileTxnLog.java
358c358,359
< next();
---
> if (!next())
> return;
447c448,450
< inputStream.close();
---
> if (inputStream != null) {
> inputStream.close();
> }

47945

No Perforce job exists for this issue.

2

33165

08/Dec/08 4:35 PM

11 years, 16 weeks ago

Reviewed

0|i061fz:

ZooKeeper

ZOOKEEPER-250

isvalidsnapshot should handle the case of 0 snapshot files better.

Bug

Closed

Major

Fixed

Mahadev Konar

08/Dec/08 13:11

13/Feb/09 16:18

11/Dec/08 16:17

3.0.0, 3.0.1

3.1.0

0

1

the isvalidsnapshot will fail with negative seek if 0 snapshot files exist. We should just return false in case the size of the snapshot is less than 5 bytes.

47946

No Perforce job exists for this issue.

2

33166

10/Dec/08 4:31 PM

11 years, 15 weeks, 6 days ago

Reviewed

0|i061g7:

ZooKeeper

ZOOKEEPER-249

QuorumPeer.getClientPort() always returns -1

Bug

Closed

Trivial

Fixed

Nitay Joffe

08/Dec/08 04:10

13/Feb/09 16:18

09/Dec/08 00:10

3.0.0, 3.0.1

3.1.0

0

1

This was changed in SVN 700690:

http://svn.apache.org/viewvc/hadoop/zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java?r1=700690&r2=700689&pathrev=700690

47947

No Perforce job exists for this issue.

1

33167

08/Dec/08 2:12 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i061gf:

ZooKeeper

ZOOKEEPER-248

QuorumPeer should use Map interface instead of HashMap implementation.

Improvement

Closed

Trivial

Fixed

Nitay Joffe

08/Dec/08 02:55

13/Feb/09 16:18

09/Dec/08 00:14

3.0.0, 3.0.1

3.1.0

quorum

0

Using HashMap directly to store quorumPeers member variable unnecessarily decides for the user what implementation of Map to use.

47948

No Perforce job exists for this issue.

1

33541

08/Dec/08 2:14 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i063rj:

ZooKeeper

ZOOKEEPER-247

fix formatting of C API in ACL section of programmer guide

Bug

Closed

Major

Fixed

Patrick D. Hunt

05/Dec/08 13:22

13/Feb/09 16:18

09/Dec/08 19:42

3.0.0, 3.0.1

3.1.0

documentation

0

1

formatting of C api is wrong in acl section of prog guide

47949

No Perforce job exists for this issue.

3

33168

09/Dec/08 5:51 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i061gn:

ZooKeeper

ZOOKEEPER-246

review error code definition in both source and docs

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

04/Dec/08 19:16

13/Feb/09 16:18

22/Dec/08 17:23

3.0.0, 3.0.1

3.1.0

c client, documentation, java client

0

ZOOKEEPER-291

A number of users have mentioned that our use of "calculated" values for error codes makes it very difficult to track error reports from the logs. In both c/java we have:

int ERROR = 100
int X_ERROR= ERROR + 1
int Y_ERROR= ERROR + 2
etc...

which makes it difficult to grep/search for "101" and find these references. A number of users have mentioned that having non-calculated values (ie X_ERROR=101) would make it easier.

I personally find calculated constants like this an anti-pattern.

We should convert these values to non-calculated at a minimum, perhaps even redefine them as enums if possible.

Also documentation, esp in the c code (the names btw c and java don't match btw) is lacking. We need to beef up docs in this area.

47950

No Perforce job exists for this issue.

3

33542

12/Dec/08 2:54 PM

11 years, 14 weeks, 2 days ago

Incompatible change, Reviewed

0|i063rr:

ZooKeeper

ZOOKEEPER-245

update readme/quickstart to be release tar, rather than source, based

Bug

Closed

Major

Fixed

Patrick D. Hunt

04/Dec/08 16:08

13/Feb/09 16:18

09/Dec/08 19:22

3.0.1

3.1.0

documentation

0

1

Owen O'Malley mentioned:
---------
we need to change both the README and quick start to assume you are working with a release instead of a source tarball. Apache releases are the approved way of getting the project. Documentation that assumes they are getting source themselves doesn't reflect that.
---------

47951

No Perforce job exists for this issue.

1

33169

09/Dec/08 7:11 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i061gv:

ZooKeeper

ZOOKEEPER-244

AsyncOpsTest fails when running consecutively

Bug

Closed

Minor

Cannot Reproduce

Unassigned

Flavio Paiva Junqueira

03/Dec/08 16:18

08/Jul/09 16:23

19/May/09 18:22

3.0.1

3.2.0

tests

0

MacOS X 10.5.5, Java 1.6.0_05

Attaching output log file.

214081

No Perforce job exists for this issue.

1

33170

11 years, 8 weeks, 5 days ago

0|i061h3:

ZooKeeper

ZOOKEEPER-243

add SEQUENCE flag documentation to the programming guide

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

03/Dec/08 13:42

13/Feb/09 16:18

09/Dec/08 17:29

3.0.0, 3.0.1

3.1.0

documentation

0

There doesn't seem to be any docs in forrest detailing the sequence flag for node creation.

In particular we should detail how the flag works, the naming format, and also rollover (when does the number roll over to 0 again).

47952

No Perforce job exists for this issue.

1

33543

09/Dec/08 5:27 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i063rz:

ZooKeeper

ZOOKEEPER-242

zookeeper c tests are faliling..

Bug

Closed

Critical

Fixed

Mahadev Konar

01/Dec/08 13:27

13/Feb/09 16:18

05/Dec/08 13:25

3.0.0

3.1.0

tests

0

1

47953

No Perforce job exists for this issue.

0

33171

11 years, 16 weeks, 6 days ago

0|i061hb:

ZooKeeper

ZOOKEEPER-241

Build of a distro fails after clean target is run

Bug

Closed

Critical

Fixed

Patrick D. Hunt

Nigel Daley

26/Nov/08 14:50

13/Feb/09 16:18

09/Dec/08 17:16

3.0.0, 3.0.1

3.1.0

build

0

1

ZOOKEEPER-205

When the ZooKeeper distro is downloaded, if the clean target is run, it deletes the .revision file:

{quote}
clean:
[delete] Deleting directory /home/foo/zookeeper-3.0.1/build
[delete] Deleting directory /home/foo/zookeeper-3.0.1/src/java/generated
[delete] Deleting directory /home/foo/zookeeper-3.0.1/src/c/generated
[delete] Deleting directory /home/foo/zookeeper-3.0.1/.revision
{quote}

This causes subsequent builds in the distro to fail with:

{quote}
compile-main:
[javac] Compiling 73 source files to /home/foo/zookeeper-3.0.1/build/classes
[javac] /home/foo/zookeeper-3.0.1/src/java/main/org/apache/zookeeper/Version.java:21: package org.apache.zookeeper.version does not exist
[javac] public class Version implements org.apache.zookeeper.version.Info{
[javac] ^
...
{quote}

47954

No Perforce job exists for this issue.

1

33172

26/Nov/08 2:52 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i061hj:

ZooKeeper

ZOOKEEPER-240

Yet another npe

Bug

Closed

Major

Fixed

Patrick D. Hunt

Flavio Paiva Junqueira

26/Nov/08 11:26

13/Feb/09 16:18

13/Jan/09 19:56

3.0.1

3.1.0

java client

0

1

java.lang.NullPointerException
at org.apache.jute.Utils.toCSVBuffer(Utils.java:234)
at org.apache.jute.CsvOutputArchive.writeBuffer(CsvOutputArchive.java:101)
at org.apache.zookeeper.proto.GetDataResponse.toString(GetDataResponse.java:48)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at org.apache.zookeeper.ClientCnxn$Packet.toString(ClientCnxn.java:230)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:586)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:626)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:852)

47955

No Perforce job exists for this issue.

0

33173

26/Nov/08 12:42 PM

11 years, 11 weeks, 1 day ago

0|i061hr:

ZooKeeper

ZOOKEEPER-239

ZooKeeper System Tests

Test

Closed

Major

Fixed

Unassigned

Benjamin Reed

24/Nov/08 16:53

26/Mar/10 13:29

03/Mar/10 13:54

3.4.0

c client, java client

0

2

ZOOKEEPER-215

It would be nice to have real system tests. Our current Java system tests start everything in the same JVM. It would be nice to be able to start up the servers on different machines.

The C tests have a system test with a script to start and stop the cluster. It would be good to extend the script and to make it used by the Java tests as well.

If we can standardize on the script, we can have script options for different cluster shells from simple ssh to torque.

47956

No Perforce job exists for this issue.

0

33544

10 years, 4 weeks, 1 day ago

0|i063s7:

ZooKeeper

ZOOKEEPER-238

HostAuthenicationProvider should be removed

Bug

Closed

Major

Fixed

Benjamin Reed

24/Nov/08 16:31

05/Sep/09 18:36

26/Aug/09 20:05

3.2.0, 3.2.1

server

0

1

I think the way the HostAuthenticationProvider is implemented could cause serious performance problems if DNS is slow or broken. The problem is that we need to do a reverse hostname resolution during connection establishment. I suggest it be removed.

47957

No Perforce job exists for this issue.

0

33174

26/Aug/09 8:04 PM

10 years, 31 weeks, 1 day ago

0|i061hz:

ZooKeeper

ZOOKEEPER-237

Add a Chroot request

New Feature

Closed

Minor

Fixed

Mahadev Konar

Benjamin Reed

24/Nov/08 16:28

08/Jul/09 16:23

24/Jun/09 19:00

3.2.0

c client, java client

0

ZOOKEEPER-146, ZOOKEEPER-390, ZOOKEEPER-424

It would be nice to be able to root ZooKeeper handles at specific points in the namespace, so that applications that use ZooKeeper can work in their own rooted subtree.

For example, if ops decides that application X can use the subtree /apps/X and application Y can use the subtree /apps/Y, X can to a chroot to /apps/X and then all its path references can be rooted at /apps/X. Thus when X creates the path "/myid", it will actually be creating the path "/apps/X/myid".

There are two ways we can expose this mechanism: 1) We can simply add a chroot(String path) API, or 2) we can integrate into a service identifier scheme for example zk://server1:2181,server2:2181/my/root. I like the second form personally.

47958

No Perforce job exists for this issue.

4

33545

24/Nov/08 4:46 PM

10 years, 40 weeks ago

0|i063sf:

ZooKeeper

ZOOKEEPER-236

ZOOKEEPER-3451 SSL Support for Atomic Broadcast protocol

Sub-task

Closed

Major

Fixed

Andor Molnar

Benjamin Reed

24/Nov/08 16:17

04/Oct/19 10:55

25/Oct/18 10:27

3.6.0, 3.5.5

quorum, security, server

4

13

0

100200

ZOOKEEPER-2750, ZOOKEEPER-3173, ZOOKEEPER-2698, ZOOKEEPER-1000

We should have the ability to use SSL to authenticate and encrypt the traffic between ZooKeeper servers. For the most part this is a very easy change. We would probably only want to support this for TCP based leader elections.

100%

100200

0

pull-request-available, ssl, ssl-tls

214080

No Perforce job exists for this issue.

0

68495

06/Mar/17 6:19 PM

1 year, 20 weeks, 2 days ago

0|i0c3ef:

ZooKeeper

ZOOKEEPER-235

SSL Support for clients

New Feature

Open

Minor

Unresolved

Unassigned

Benjamin Reed

24/Nov/08 16:15

02/Mar/17 18:32

c client, java client, server

11

18

ZOOKEEPER-2125, ZOOKEEPER-2122

ZooKeeper should be able to support SSL for ZooKeeper clients. As part of the implementation we should also add an X509AuthenticationProvider so that client side certifications can be used for authentication.

The tricky part of the implementation will be integrating with the non-blocking NIO calls that we use. There are various web pages that describe how to do it.

214079

No Perforce job exists for this issue.

0

68496

09/Dec/15 12:55 PM

3 years, 3 weeks ago

0|i0c3en:

ZooKeeper

ZOOKEEPER-234

Eliminate using statics to initialize the sever. Should allow server to be more embeddable in OSGi enviorments.

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Hiram R. Chirino

24/Nov/08 16:01

13/Feb/09 16:18

15/Jan/09 17:59

3.1.0

server

0

Patrick request I open up this in issue in this [email thread|http://n2.nabble.com/ActiveMQ-is-now-using-ZooKeeper-td1573272.html]

The main culprit I've noticed is:
{code}
ServerStats.registerAsConcrete();
{code}

But there may be others.

47959

No Perforce job exists for this issue.

2

33546

08/Dec/08 1:53 PM

11 years, 10 weeks, 6 days ago

Reviewed

0|i063sn:

ZooKeeper

ZOOKEEPER-233

Create a slimmer jar for clients to reduce their disk footprint.

New Feature

Open

Trivial

Unresolved

Unassigned

Hiram R. Chirino

24/Nov/08 15:56

14/Dec/19 06:09

3.7.0

build, java client

3

8

ZOOKEEPER-1275

ZOOKEEPER-652, ZOOKEEPER-278, ZOOKEEPER-1003

Patrick request I open up this in issue in this [email thread|http://n2.nabble.com/ActiveMQ-is-now-using-ZooKeeper-td1573272.html]

62831

No Perforce job exists for this issue.

0

68497

24/Nov/08 4:52 PM

5 years, 29 weeks, 6 days ago

not a blocker for 3.2, moving to 3.3

0|i0c3ev:

ZooKeeper

ZOOKEEPER-232

testablezookeeper file is missing apache licence headers.

Bug

Closed

Blocker

Fixed

Mahadev Konar

24/Nov/08 14:48

13/Feb/09 16:18

24/Nov/08 15:50

3.0.1, 3.1.0

0

1

47960

No Perforce job exists for this issue.

1

33175

24/Nov/08 3:45 PM

11 years, 18 weeks, 2 days ago

Reviewed

0|i061i7:

ZooKeeper

ZOOKEEPER-231

Quotas in zookeeper

New Feature

Closed

Major

Fixed

Mahadev Konar

21/Nov/08 16:13

13/Feb/09 16:18

03/Feb/09 00:26

3.0.0

3.1.0

0

1

ZOOKEEPER-119

creating quota's in zookeeper so that a runaway application does not bring down the zookeeper cluster.

47961

No Perforce job exists for this issue.

9

33547

25/Nov/08 1:54 PM

11 years, 8 weeks, 2 days ago

Reviewed

0|i063sv:

ZooKeeper

ZOOKEEPER-230

Improvements to FLE

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

20/Nov/08 09:37

13/Feb/09 16:18

12/Dec/08 15:22

3.0.0

3.1.0

leaderElection

0

I'm about to attach a patch that implements the following modifications:

. Currently, if a server is on leader election and doesn't receive a notification for some amount of time t, then it sends a new set of notifications if at least one server has delivered a message from the previous set. With this patch, the amount of time a server waits for a notification before sending a new set increases exponentially;
. I have separated connecting to servers and queuing new notification messages. Before they were all in the same message. The advantage is that now I can tell to an instance of QuorumCnxManager to try to connect to other servers without generating new notification messages;
. I have changed the logging level of several messages on QuorumCnxManager. They were "warn", but they should really be either "info" or "debug". I've changed them to info.

47962

No Perforce job exists for this issue.

5

33548

20/Nov/08 1:02 PM

11 years, 15 weeks, 2 days ago

Reviewed

0|i063t3:

ZooKeeper

ZOOKEEPER-229

improve documentation regarding user's responsibility to cleanup datadir (snaps/logs)

Improvement

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

18/Nov/08 16:54

13/Feb/09 16:18

30/Jan/09 20:23

3.0.0

3.1.0

documentation, server

0

ZOOKEEPER-252

It's not very obvious in the docs that the operator of the zk server is responsible for managing the files in the datadir:

http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperAdmin.html#File+Management

specifically removing the old snaps/transactionallogs

we need to make this more obvious, perhaps call it out in the quickstart (note pointing to this document)

Perhaps we should include example script and cron as well (docs)

47963

No Perforce job exists for this issue.

2

33549

28/Jan/09 7:30 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i063tb:

ZooKeeper

ZOOKEEPER-228

apache header missing in DBtest

Bug

Closed

Blocker

Fixed

Mahadev Konar

18/Nov/08 15:02

13/Feb/09 16:18

18/Nov/08 15:13

3.0.0

3.0.1, 3.1.0

0

1

one of the test files is missing apache headers ...

47964

No Perforce job exists for this issue.

1

33176

18/Nov/08 3:08 PM

11 years, 19 weeks ago

Reviewed

0|i061if:

ZooKeeper

ZOOKEEPER-227

gcc warning from recordio.h

Bug

Closed

Minor

Fixed

Chris Darroch

18/Nov/08 14:50

13/Feb/09 16:18

19/Nov/08 15:54

3.0.0

3.0.1, 3.1.0

c client

0

1

Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific "void". The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h.

47965

No Perforce job exists for this issue.

1

33177

18/Nov/08 3:03 PM

11 years, 19 weeks ago

Reviewed

0|i061in:

ZooKeeper

ZOOKEEPER-226

exists calls fails on server if node has null data

Bug

Closed

Blocker

Fixed

Mahadev Konar

Patrick D. Hunt

18/Nov/08 14:27

13/Feb/09 16:18

18/Nov/08 18:46

3.0.0

3.0.1, 3.1.0

server

0

1

An NPE will be generated on the server (and resulting in client getting MarshallingException) if exists() is called on a node with null data.

workaround is to create the node with non-null data.

be sure to update the tests for this case

47966

No Perforce job exists for this issue.

1

33178

18/Nov/08 2:49 PM

11 years, 19 weeks ago

Reviewed

0|i061iv:

ZooKeeper

ZOOKEEPER-225

c client should log an info message in zookeeper_init detailing connection parameters

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

18/Nov/08 13:53

13/Feb/09 16:18

12/Dec/08 19:27

3.0.0, 3.0.1

3.1.0

c client

0

I've debugged a few situations where the client connection param was not set properly (even though the user swore that it was). We should log info level message during init to enable us (and user) to easier debug.

47967

No Perforce job exists for this issue.

2

33550

10/Dec/08 4:02 PM

11 years, 15 weeks, 2 days ago

Reviewed

0|i063tj:

ZooKeeper

ZOOKEEPER-224

Deploy ZooKeeper jars/artifacts to a Maven Repository

Task

Closed

Critical

Fixed

Patrick D. Hunt

Hiram R. Chirino

17/Nov/08 20:02

01/Jun/11 03:22

19/Mar/10 14:15

3.0.0

3.3.0

build

4

10

HBASE-1933, ZOOKEEPER-537, HIVE-1235, ZOOKEEPER-529, ZOOKEEPER-1085

I've created the maven poms needed for the 3.0.0 release.

The directory structure and artifacts located at:
http://people.apache.org/~chirino/zk-repo/
aka
people.apache.org:/x1/users/chirino/public_html/zk-repo

Just need sto get GPG signed by the project KEY and deployed to:
people.apache.org:/www/people.apache.org/repo/m2-ibiblio-rsync-repository

Who's the current ZooKeeper release manager?

47968

No Perforce job exists for this issue.

0

33551

17/Nov/08 8:12 PM

10 years, 1 week, 4 days ago

Reviewed

0|i063tr:

ZooKeeper

ZOOKEEPER-223

default log4j root logger configuration has neg perf impact with no benefit, change default level to INFO

Improvement

Closed

Major

Fixed

Patrick D. Hunt

14/Nov/08 02:10

13/Feb/09 16:18

17/Nov/08 11:14

3.0.0

3.0.1, 3.1.0

java client, server

0

The default log4j.properties has:

log4j.rootLogger=DEBUG, CONSOLE

where the console appender is only outputting INFO level and above.

This is resulting in a large number of debug logs being generated, which are all filtered out by the console appender (by default there are no other appenders used)

We need to change the default rootLogger to INFO level, which will resolve this issue.

Users are free to change this of course -- most likely they would only do this when debugging an issue, and would not want to run in DEBUG in production (again, they are free to choose).

47969

No Perforce job exists for this issue.

1

33552

17/Nov/08 11:08 AM

11 years, 19 weeks ago

Reviewed

0|i063tz:

ZooKeeper

ZOOKEEPER-222

print C client log message timestamp in human readable form

Improvement

Closed

Minor

Fixed

Patrick D. Hunt

13/Nov/08 17:11

13/Feb/09 16:18

15/Dec/08 17:36

3.0.0

3.1.0

c client

0

Noticed this issue during a debugging session -- it's difficult to read/compare the timestamps from the c client.

The C client timestamp should be formatted with the same format as the JAVA log timestamps.

Note: java uses ISO8601 timestamp formatting by default.

Perhaps we should just bit the bullet and move to log4c? Consider this option when addressing this issue.

47970

No Perforce job exists for this issue.

1

33553

15/Dec/08 5:36 PM

11 years, 15 weeks, 2 days ago

Reviewed

0|i063u7:

ZooKeeper

ZOOKEEPER-221

log message in ZkWatchManager.materialize missing conditional

Bug

Closed

Major

Fixed

Patrick D. Hunt

12/Nov/08 18:21

13/Feb/09 16:18

19/Nov/08 05:24

3.0.0

3.0.1, 3.1.0

java client

0

2

ZOOKEEPER-204

the following code is missing conditional log call based on result of remove call (size > 0)

// XXX This shouldn't be needed, but just in case
> synchronized (existWatches) {
> addTo(existWatches.remove(path), result);
> LOG.warn("We are triggering an exists watch for delete! Shouldn't happen!");
> }
>

47971

No Perforce job exists for this issue.

1

33179

18/Nov/08 6:17 PM

11 years, 19 weeks, 1 day ago

0|i061j3:

ZooKeeper

ZOOKEEPER-220

programming guide watches section should clarify server/clientlib role in data/child watch maint.

Improvement

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

12/Nov/08 14:02

13/Feb/09 16:18

02/Feb/09 18:01

3.1.0

documentation

0

http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatchesonly

http://hadoop.apache.org/zookeeper/docs/r3.0.0/zookeeperProgrammers.html#ch_zkWatches

The server is maintaining two lists, it's not really clear in some sections that it's the server, esp as in v3 the client is also maintaining it's own list(s).

47972

No Perforce job exists for this issue.

2

33554

02/Feb/09 5:15 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i063uf:

ZooKeeper

ZOOKEEPER-219

events.poll timeout in watcher test too short

Bug

Closed

Minor

Fixed

Patrick D. Hunt

11/Nov/08 17:01

13/Feb/09 16:18

17/Nov/08 19:13

3.0.0

3.0.1, 3.1.0

0

1

in watchertest there are some event.poll calls that have 1milli timeouts

e = localWatcher.events.poll(1, TimeUnit.MILLISECONDS);

this is showing falure in some cases under hudson (I assume when it's under load from other tests running for other proj)

We should review the poll calls and verify adequate timeouts.

47973

No Perforce job exists for this issue.

1

33180

17/Nov/08 7:09 PM

11 years, 19 weeks ago

Reviewed

0|i061jb:

ZooKeeper

ZOOKEEPER-218

error in barrier recipe example code

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Nov/08 17:24

13/Feb/09 16:18

18/Nov/08 18:14

3.0.0

3.0.1, 3.1.0

documentation

0

1

Report from user Murali Vemulapati. The wiki recipe page also should be updated to handle multiple clients using the barrier on the same system (multiple processes, etc...). Currently the docs/example support only a single user of the barrier per host.

------------------

I believe there is a typo in the barrier example given at:

http://hadoop.apache.org/zookeeper/docs/current/zookeeperTutorial.html

With the following fix, the program runs as expected:
==============
83c83
< this.name = new String(InetAddress.getLocalHost().getCanonicalHostName().toString());
---
> name = new String(InetAddress.getLocalHost().getCanonicalHostName().toString());
100c100
< CreateMode.EPHEMERAL_SEQUENTIAL);
---
> CreateMode.EPHEMERAL);
==============
The first change assigns the name to the instance variable 'name' of Barrier class (otherwise the 'name' instance variable will have a value of 'null'
when calling zk.create to create the child node under the root barrier node).
The second change lets us run multiple processes on the same machine.

thanks
murali

47974

No Perforce job exists for this issue.

1

33181

18/Nov/08 6:07 PM

11 years, 19 weeks ago

Reviewed

0|i061jj:

ZooKeeper

ZOOKEEPER-217

handle errors when parsing config file, throw illegalargumentexception rather than exit()

Improvement

Closed

Minor

Fixed

Mahadev Konar

Flavio Paiva Junqueira

05/Nov/08 12:50

01/May/13 22:29

18/Nov/08 12:24

3.0.0

3.0.1, 3.1.0

server

0

ZOOKEEPER-209

Discussing 209 with Ben today, we thought that it would be better to have the parse method of QuorumPeerConfig returning a boolean that indicates whether the configuration is good or not, and let the caller decide whether to exit or not. Currently we execute a System.exit() on QuorumPeerConfig.parse when we have a critical configuration error.

47975

No Perforce job exists for this issue.

2

33555

17/Nov/08 5:20 PM

11 years, 19 weeks ago

Reviewed

0|i063un:

ZooKeeper

ZOOKEEPER-216

Improve logging in C client

Improvement

Resolved

Major

Won't Fix

Unassigned

Patrick D. Hunt

04/Nov/08 20:00

23/Apr/14 16:17

3.0.0

3.5.0

c client

0

1

While debugging a client issue we saw multiple areas for improvement in C logging.

It might be a good idea to also move to log4c. Anyone have experience with this? multithread an issue?

70748

No Perforce job exists for this issue.

0

68498

05/Feb/09 1:50 PM

5 years, 48 weeks, 1 day ago

not a blocker for 3.2, moving to 3.3

0|i0c3f3:

ZooKeeper

ZOOKEEPER-215

expand system test environment

New Feature

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

04/Nov/08 19:57

13/Feb/09 16:18

30/Jan/09 20:04

3.1.0

tests

0

ZOOKEEPER-149, ZOOKEEPER-239

Currently our system tests are lumped in with our unit tests. It would be great to have a system test environment where we could run larger scale testing. Say you have 20 hosts, and you would like to test a serving ensemble with 7 servers and 100 clients running particular operations. It should be easy to test this scenario. Additionally during the test it should be possible to simulate serving node failure, etc...

I've had a brief conversation with Ben about this and he's going to take this JIRA.

47976

No Perforce job exists for this issue.

6

33556

25/Jan/09 1:06 AM

11 years, 8 weeks, 3 days ago

Reviewed

0|i063uv:

ZooKeeper

ZOOKEEPER-214

add new "stat reset" command to server admin port

Improvement

Closed

Major

Fixed

Patrick D. Hunt

04/Nov/08 19:55

13/Feb/09 16:18

23/Dec/08 16:29

3.1.0

server

0

A new command should be added to the server admin port (client port) that allows users to reset the stats.

We saw cases during troubleshooting where we would have liked to reset the "min/avg/max latency" stats, but were unable to do so (short of restarting the server).

47977

No Perforce job exists for this issue.

2

33557

22/Dec/08 8:31 PM

11 years, 14 weeks, 1 day ago

Reviewed

0|i063v3:

ZooKeeper

ZOOKEEPER-213

programmer guide C api docs are out of sync with latest zookeeper.h

Bug

Closed

Major

Fixed

Patrick D. Hunt

29/Oct/08 02:02

13/Feb/09 16:18

17/Nov/08 18:51

3.0.0

3.0.1, 3.1.0

documentation

0

1

see
http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#Zookeeper+C+client+API
and compare with src/c/zookeeper.h

47978

No Perforce job exists for this issue.

1

33182

17/Nov/08 6:27 PM

11 years, 19 weeks ago

Reviewed

0|i061jr:

ZooKeeper

ZOOKEEPER-212

snapshot is synchronous in 3.0

Bug

Closed

Critical

Fixed

Mahadev Konar

28/Oct/08 17:54

13/Feb/09 16:18

17/Nov/08 18:22

3.0.0

3.0.1, 3.1.0

0

1

the snapshot in 3.0 is syunchronous. this will affect performance of the system.

47979

No Perforce job exists for this issue.

3

33183

31/Oct/08 11:11 AM

11 years, 19 weeks ago

Reviewed

0|i061jz:

ZooKeeper

ZOOKEEPER-211

Not all Mock tests are working

Test

Closed

Minor

Fixed

Benjamin Reed

28/Oct/08 14:14

13/Feb/09 16:18

18/Nov/08 20:28

3.0.1, 3.1.0

c client

0

For the 3.0 release there was a bug in the ZKMock code that prevented some tests from working. This patch fixes the bug and reenables the tests.

47980

No Perforce job exists for this issue.

1

33558

28/Oct/08 2:25 PM

11 years, 19 weeks ago

Reviewed

0|i063vb:

ZooKeeper

ZOOKEEPER-210

Require Java 6

Task

Closed

Major

Fixed

Patrick D. Hunt

27/Oct/08 21:35

13/Feb/09 16:18

22/Jan/09 10:29

3.1.0

documentation, java client, jmx, server

0

We should require Java 6 for release 3.1.0. At this point there is full java6 support on all major platforms (linux/windows/macosx).
The vast majority of ZooKeeper contributors use Java 6 when developing, and hence do run into (usually minor) back-compatibility problems.
I believe there are also some benefits regarding JMX (updated spec support in java6) if we drop java 5.

FWIW: Hadoop Core has also moved to require Java6:
https://issues.apache.org/jira/browse/HADOOP-2325

47981

No Perforce job exists for this issue.

1

33559

28/Oct/08 2:49 PM

11 years, 9 weeks, 6 days ago

ZooKeeper now requires Java 6.

Incompatible change, Reviewed

0|i063vj:

ZooKeeper

ZOOKEEPER-209

nullpointerexception if election port is not specified.

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Mahadev Konar

27/Oct/08 19:22

01/May/13 22:29

18/Nov/08 12:25

3.0.0

3.0.1, 3.1.0

0

1

ZOOKEEPER-217

The quoruom servers throw a nullpointer exception and still keep running. We should atleast have a nice debug message and quit...

47982

No Perforce job exists for this issue.

1

33184

05/Nov/08 12:40 PM

11 years, 19 weeks, 2 days ago

Reviewed

0|i061k7:

ZooKeeper

ZOOKEEPER-208

Zookeeper C client uses API that are not thread safe, causing crashes when multiple instances are active

Bug

Closed

Critical

Fixed

Austin Bennett

27/Oct/08 01:31

13/Feb/09 16:18

19/Nov/08 15:49

3.0.0

3.0.1, 3.1.0

c client

0

1

Linux

The Zookeeper C client library uses gethostbyname and strtok, both of which are not safe to use from multiple threads.

The problem is resolved by using getaddrinfo and strtok_r in place of the older API.

47983

No Perforce job exists for this issue.

3

33185

27/Oct/08 2:55 AM

11 years, 19 weeks ago

Reviewed

0|i061kf:

ZooKeeper

ZOOKEEPER-207

All the threads should have names so that its easier to see throguh a stack trace.

Improvement

Open

Minor

Unresolved

Mahadev Konar

24/Oct/08 17:38

14/Dec/19 06:07

3.0.0

3.7.0

server

0

2

ZOOKEEPER-1907

the threads should have names so that its easier to look through the stack trace.

70792

No Perforce job exists for this issue.

0

68499

24/Oct/08 7:02 PM

10 years, 3 weeks, 3 days ago

not a blocker for 3.2, moving to 3.3

0|i0c3fb:

ZooKeeper

ZOOKEEPER-206

documentation tab should contain the version number (and other small site changes)

Bug

Closed

Minor

Fixed

Patrick D. Hunt

23/Oct/08 15:20

13/Feb/09 16:18

18/Nov/08 18:36

3.0.0

3.0.1, 3.1.0

documentation

0

1

Feedback from Doug Cutting on 3.0.0 documentation:

- The "Zookeeper Documentation" tab should contain the version number.
- "Informal Documentation" might better be named "Other Documentation".
- The "Other Info" page should be removed, since it contains nothing.

47984

No Perforce job exists for this issue.

2

33186

18/Nov/08 6:20 PM

11 years, 19 weeks ago

Reviewed

0|i061kn:

ZooKeeper

ZOOKEEPER-205

Error on version-info when compiling from the tarball distribution

Bug

Closed

Minor

Duplicate

Mahadev Konar

Flavio Paiva Junqueira

22/Oct/08 17:46

13/Feb/09 16:18

05/Dec/08 14:48

3.0.0

3.1.0

0

ZOOKEEPER-241

When running ant with the code from the release tarball, I get the following messages and stack trace:

{noformat}
svn-revision:
[exec] svn: '.' is not a working copy

version-info:
[java] All version-related parameters must be valid integers!
[java] Exception in thread "main" java.lang.NumberFormatException: For input string: ""
[java] at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
[java] at java.lang.Integer.parseInt(Integer.java:468)
[java] at java.lang.Integer.parseInt(Integer.java:497)
[java] at org.apache.zookeeper.version.util.VerGen.main(VerGen.java:111)
[java] Java Result: 1
{noformat}

This seems to be because the code is not coming from svn, so it can't find the version information.

214078

No Perforce job exists for this issue.

1

33187

22/Oct/08 10:26 PM

11 years, 16 weeks, 6 days ago

0|i061kv:

ZooKeeper

ZOOKEEPER-204

SetWatches needs to be the first message after auth messages to the server

Bug

Closed

Major

Fixed

Benjamin Reed

22/Oct/08 02:06

13/Feb/09 16:18

19/Nov/08 05:23

3.0.0

3.0.1, 3.1.0

java client

0

1

ZOOKEEPER-221

When the ZooKeeper java client makes a connection it queues a SetWatches request. The problem is that it puts the request at the end of the outgoing requests. This means that watches for requests that were queued before the connection is made and after the disconnect may improperly get their watches triggered.

47985

No Perforce job exists for this issue.

4

33188

27/Oct/08 5:36 PM

11 years, 19 weeks ago

Reviewed

0|i061l3:

ZooKeeper

ZOOKEEPER-203

fix datadir typo in releasenotes

Bug

Closed

Minor

Fixed

Patrick D. Hunt

21/Oct/08 23:43

25/Oct/08 21:10

21/Oct/08 23:47

3.0.0

documentation

0

1

typo in releasenotes note on datalog/data dirs.

47986

No Perforce job exists for this issue.

1

33189

21/Oct/08 11:46 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061lb:

ZooKeeper

ZOOKEEPER-202

Phantom ephemeral node

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

21/Oct/08 17:24

26/Mar/10 13:28

22/Oct/08 13:17

3.0.0

0

1

One of our users has observed that an ephemeral znode had gone away once its creator had disconnected according to the leader, but one follower believed that it existed long after the znode had been deleted. Apparently the follower was never going to delete it. Because the leader wouldn't recognize the znode as an existing one, any attempt to delete the znode failed. We have to investigate if this is related to any known bug, although, to my knowledge, this is the first time it happens. It is important to note that the user was running an older version of our code.

47987

No Perforce job exists for this issue.

0

33190

21/Oct/08 5:45 PM

11 years, 23 weeks, 1 day ago

0|i061lj:

ZooKeeper

ZOOKEEPER-201

validate magic number when reading snapshot and transaction logs

Bug

Closed

Blocker

Fixed

Mahadev Konar

Patrick D. Hunt

21/Oct/08 00:55

25/Oct/08 21:10

21/Oct/08 14:55

3.0.0

server

0

1

The snapshot and transaction log files are not validating the magic numbers when read.

Mahadev, can you update the code and tests for this? Possible for 3.0 or wait post 3.0? (feel free to fix now or reassign version)

Please add tests for this.

47988

No Perforce job exists for this issue.

1

33191

21/Oct/08 2:15 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061lr:

ZooKeeper

ZOOKEEPER-200

the magic number for snapshot and log must be different (currently same)

Bug

Closed

Major

Fixed

Patrick D. Hunt

21/Oct/08 00:39

25/Oct/08 21:10

21/Oct/08 13:30

3.0.0

server

0

1

the magic number for the snapshot and transaction logs are currently the same - they should be different. Also the magic numbers should also be more indicative of the type of file (currently "AK47" for both, not very useful in determining type of file)

47989

No Perforce job exists for this issue.

2

33192

21/Oct/08 1:08 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061lz:

ZooKeeper

ZOOKEEPER-199

fix log messages in persistence code

Bug

Closed

Minor

Fixed

Mahadev Konar

Patrick D. Hunt

20/Oct/08 19:59

25/Oct/08 21:10

21/Oct/08 00:36

3.0.0

server

0

1

there are 2 log messages during server startup that are misleading:

2008-10-20 16:36:41,135 - INFO [main:FileTxnLog$FileTxnIterator@441] - EOF excepton java.io.EOFException: Failed to read

2008-10-20 16:36:41,120 - ERROR [main:FileTxnSnapLog@114] - 2(higestZxid) >= 2(next log) for type 1

the first log should be DEBUG and should be changed to say something like eof reached in <file>, reading next file

while the second log seems to indicate an error - however the server is still starting. either this is not an error, or it is an error and it's not being handled correctly.

47990

No Perforce job exists for this issue.

1

33193

20/Oct/08 8:57 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061m7:

ZooKeeper

ZOOKEEPER-198

apache license header missing from FollowerSyncRequest.java

Bug

Closed

Minor

Fixed

Patrick D. Hunt

20/Oct/08 16:46

25/Oct/08 21:10

20/Oct/08 17:09

3.0.0

server

0

1

add license to file & run the rat tool to verify svn

47991

No Perforce job exists for this issue.

1

33194

20/Oct/08 4:57 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061mf:

ZooKeeper

ZOOKEEPER-197

create checksums for snapshots

Improvement

Closed

Blocker

Fixed

Mahadev Konar

20/Oct/08 16:29

25/Oct/08 21:10

21/Oct/08 00:26

3.0.0

0

ZOOKEEPER-38 added checksums for txn logs. We need to add similar checksums for snapshots.

47992

No Perforce job exists for this issue.

2

33560

21/Oct/08 12:26 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063vr:

ZooKeeper

ZOOKEEPER-196

doxygen comment for state argument of watcher_fn typedef and implementation differ ("...one of the *_STATE constants, otherwise -1")

Bug

Closed

Major

Fixed

Benjamin Reed

Maxim P. Dementiev

20/Oct/08 08:12

08/Jul/09 16:23

08/Jun/09 17:31

3.2.0

c client

0

1

Linux

In zookeeper.h:
* \param state connection state. If the type is ZOO_SESSION_EVENT, the state value
* will be one of the *_STATE constants, otherwise -1.
but for this sequence:
1. zoo_awexists(name)
2. zoo_acreate(name)
we've got a watcher callback with type=ZOO_CREATED_EVENT and state!=-1

I think the comment should be altered to underline the difference between zookeeper_init() callback usage and others ("the getter API functions with the "w" prefix in their names") for the new "watcher object" style.
It looks like the type and path argument values are useless for the former (because type is always ZOO_SESSION_EVENT, and path is always empty), and the state is useless for the latter (it is considered to be -1).

And more, the state of the legacy style should be commented - will it be marked as obsolete? Or will it be supported in the future?

I wonder if there are any plans to split current watcher_fn callback to something like:
1. new watcher_fn: typedef void (*watcher_fn)(zhandle_t *zh, int type, const char *path, void *watcherCtx);
2. connection_fn: typedef void (*watcher_fn)(zhandle_t *zh, int state, void *context);
Because, you see, the usage is different and there is no any common set of arguments apart from zh (which is common for API) and context.

47993

No Perforce job exists for this issue.

1

33195

27/Jan/09 5:58 PM

10 years, 42 weeks ago

Reviewed

0|i061mn:

ZooKeeper

ZOOKEEPER-195

Configuration information is spread across too many docs. Consolidate into one

Improvement

Open

Minor

Unresolved

Unassigned

Robbie Scott

16/Oct/08 20:27

14/Dec/19 06:07

3.0.0

3.7.0

documentation

0

1

3600

0%

There are definition lists of the configuration parameters in both the getting started guide and in the admin guide. It should probably only exist in the administration guide in the configuration parameters section.

Note that in the getting started guide, definitions of config params can be found in both

- Installing and Running ZooKeeper in Single Server Mode
- Running Replicated ZooKeeper

0%

3600

66776

No Perforce job exists for this issue.

1

68500

16/Oct/08 8:48 PM

8 years, 35 weeks, 1 day ago

not a blocker for 3.2, moving to 3.3

0|i0c3fj:

ZooKeeper

ZOOKEEPER-194

Fix terminology in zookeeperAdmin.xml

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

16/Oct/08 11:32

25/Oct/08 21:10

16/Oct/08 18:31

3.0.0

documentation

0

According to the trunk version of zookeeperAdmin.xml, a quorum is a group of servers running an instance of the zookeeper service.This terminology is not standard. We should use "ensemble" to denote a group of zookeeper servers, and "quorum" to denote a minimum subset of the ensemble necessary, for example, to commit an operation or to support a leader. In our case, a quorum is always a majority of servers of the ensemble.

47994

No Perforce job exists for this issue.

1

33561

16/Oct/08 6:31 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063vz:

ZooKeeper

ZOOKEEPER-193

update java example doc to compile with latest zookeeper

Bug

Closed

Major

Fixed

Patrick D. Hunt

15/Oct/08 20:14

25/Oct/08 21:10

16/Oct/08 16:09

3.0.0

documentation

0

1

In particular the pkg structure has changed.

47995

No Perforce job exists for this issue.

2

33196

16/Oct/08 10:29 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061mv:

ZooKeeper

ZOOKEEPER-192

trailing whitespace in config file can cause number format exceptions

Bug

Closed

Minor

Fixed

Patrick D. Hunt

15/Oct/08 18:49

08/Jul/09 16:23

22/May/09 15:02

3.0.0

3.2.0

server

0

1

a config file with trailing whitespace can cause number format exception

for example a line such as

clientPort=2181

where 2181 is followed by a space character, will fail to parse with number format excetion "2181 " (notice the space).

We need to trim whitespace when parsing numbers

47996

No Perforce job exists for this issue.

1

33197

21/May/09 9:34 PM

10 years, 44 weeks, 5 days ago

Reviewed

0|i061n3:

ZooKeeper

ZOOKEEPER-191

forrest docs for upgrade.

Improvement

Closed

Blocker

Fixed

Mahadev Konar

15/Oct/08 18:05

25/Oct/08 21:10

21/Oct/08 20:14

3.0.0

documentation

0

As soon as ZOOKEEPER-5 gets checked in we will need forrest docs for upgrade feature - how to rollback etc. etc..

47997

No Perforce job exists for this issue.

2

33562

21/Oct/08 8:09 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063w7:

ZooKeeper

ZOOKEEPER-190

Reorg links to docs and navs to docs into related sections

Bug

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

14/Oct/08 19:42

25/Oct/08 21:10

15/Oct/08 00:05

3.0.0

documentation

0

1

7200

0%

Right now, on the main documentation page, and also in the left-hand nav, the docs are in a single flat list. Let's reorganize them into related groups. Maybe, for example, something like this:

Overview
- Overview
- Getting Started

Developer Docs
- Programmer's Guide
- Recipes
- Java Example
- Barrier and Queues Tutorial
- API

Administrtor Docs
- Administrators Guide

Contributor Docs
- Zookeeper Internals

Miscellaneous
- FAQ
- Wiki
- Other Information

0%

7200

47998

No Perforce job exists for this issue.

1

33198

15/Oct/08 12:05 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061nb:

ZooKeeper

ZOOKEEPER-189

forrest build not validated xml of input documents

Bug

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

14/Oct/08 18:39

25/Oct/08 21:10

14/Oct/08 19:45

3.0.0

documentation

0

1

1800

0%

It's possible to build the documentation without validating the XML of the source document.

0%

1800

47999

No Perforce job exists for this issue.

1

33199

14/Oct/08 7:45 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061nj:

ZooKeeper

ZOOKEEPER-188

Check that election port is present for all servers

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

14/Oct/08 16:08

25/Oct/08 21:10

14/Oct/08 17:10

3.0.0

0

Currently, if we use the default leader election, but we do not provide an election port for every server, then zookeeper servers simply crash. I'm attaching a patch to output an error message warning the user of the configuration problem. It might be a better idea to through an exception upon fatal configuration problems, but QuorumPeerConfig.parse() currently throws no exception, so we might postpone such a change until after the 3.0.0 release.

48000

No Perforce job exists for this issue.

1

33563

14/Oct/08 5:10 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063wf:

ZooKeeper

ZOOKEEPER-187

CreateMode api docs missing

Bug

Closed

Minor

Fixed

Patrick D. Hunt

14/Oct/08 14:52

25/Oct/08 21:10

15/Oct/08 12:17

3.0.0

documentation, java client

0

1

End user API docs are missing for CreateMode. (dev api docs are fine)

48001

No Perforce job exists for this issue.

1

33200

15/Oct/08 2:04 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061nr:

ZooKeeper

ZOOKEEPER-186

add new "releasenotes.xml" to forrest documentation

Task

Closed

Major

Fixed

Patrick D. Hunt

14/Oct/08 12:34

25/Oct/08 21:10

15/Oct/08 12:16

3.0.0

documentation

0

ZOOKEEPER-145

we need releasenotes versioned docs similar to what hadoop has (see their svn, docs, and "howtorelease" twiki)

48002

No Perforce job exists for this issue.

2

33564

15/Oct/08 2:21 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063wn:

ZooKeeper

ZOOKEEPER-185

Improved version of FLETest

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

13/Oct/08 19:21

25/Oct/08 21:10

14/Oct/08 12:58

3.0.0

leaderElection

0

FLETest sometimes fails because we don't run it along with all other parts of ZooKeeper. I'm attaching a patch that modifies the current implementation of FLETest and simulates the expected behavior of other parts of ZooKeeper when running along with them.

48003

No Perforce job exists for this issue.

3

33565

13/Oct/08 8:33 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063wv:

ZooKeeper

ZOOKEEPER-184

tests: An explicit include derective is needed for the usage of memcpy(), memset(), strlen(), strdup() and free() functions.

Bug

Closed

Major

Fixed

Maxim P. Dementiev

13/Oct/08 07:52

25/Oct/08 21:10

14/Oct/08 16:29

3.0.0

c client

0

1

Linux

Some compilation environments provide implicit inclusion of certain system headers.
But any way it's not a reason to exploit it in platform-independent projects.

TestHashtable.cc and LibCMocks.h from src/c/tests/ use those functions without including corresponding system headers.

Modern versions of GCC are very strict.
You cannot build the code like this with the help of GCC version 4.3.

48004

No Perforce job exists for this issue.

1

33201

14/Oct/08 4:29 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061nz:

ZooKeeper

ZOOKEEPER-183

Array subscript is above array bounds in od_completion(), src/cli.c.

Bug

Closed

Major

Fixed

Maxim P. Dementiev

10/Oct/08 12:05

25/Oct/08 21:10

14/Oct/08 16:17

3.0.0

c client

0

1

Linux

Having:
char buf[4096*16];
Present:
buf[sizeof(buf)]=0;
Must be:
buf[sizeof(buf)-1]=0;

48005

No Perforce job exists for this issue.

1

33202

10/Oct/08 2:32 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061o7:

ZooKeeper

ZOOKEEPER-182

zookeeper_init accepts empty host-port string and returns valid pointer to zhandle_t.

Bug

Closed

Major

Fixed

Maxim P. Dementiev

10/Oct/08 11:49

25/Oct/08 21:10

14/Oct/08 16:10

3.0.0

c client

0

1

Linux

Please, add this test to src/c/tests/TestZookeeperInit.cc to reproduce this:

void testEmptyAddressString()
{
zh=zookeeper_init("",0,0,0,0,0);
CPPUNIT_ASSERT(zh==0);
}

48006

No Perforce job exists for this issue.

2

33203

12/Oct/08 11:45 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061of:

ZooKeeper

ZOOKEEPER-181

Some Source Forge Documents did not get moved over: javaExample, zookeeperTutorial, zookeeperInternals

Bug

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

09/Oct/08 11:45

25/Oct/08 21:10

10/Oct/08 09:58

3.0.0

0

1

172800

0%

ZOOKEEPER-73

Some Source Forge Documents did not get moved over:

javaExample
zookeeperTutorial
(These two I will move over directly)

zookeeperLogging
zookeeperAtomicBroadcast
(These two I will roll up into a new document: zookeeperInternals)

We can expand zookeeperInterals as new sections are needed, maybe one day morphing it into the "complete guide for zookeeper contributers"

0%

172800

48007

No Perforce job exists for this issue.

3

33204

09/Oct/08 1:24 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061on:

ZooKeeper

ZOOKEEPER-180

ZOOKEEPER-160 Placeholder sections needed in document for new topics that the umbrella jira discusses

Sub-task

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

07/Oct/08 20:08

25/Oct/08 21:10

08/Oct/08 14:50

3.0.0

documentation

0

The admin doc needs some sections so contributors can add the new topics: designing, provisioning, etc.

48008

No Perforce job exists for this issue.

2

33566

08/Oct/08 2:49 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063x3:

ZooKeeper

ZOOKEEPER-179

ZOOKEEPER-168 need new sections for basic operations content

Sub-task

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

07/Oct/08 19:40

25/Oct/08 21:10

08/Oct/08 14:18

3.0.0

0

The zookeeperProgrammers.xml, which is the Programmer's Guide, needs new sections for each of the new topics that the JIRA talks about. For now they can be empty.

48009

No Perforce job exists for this issue.

1

33567

08/Oct/08 2:18 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063xb:

ZooKeeper

ZOOKEEPER-178

FLE test.

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Mahadev Konar

06/Oct/08 16:50

25/Oct/08 21:10

10/Oct/08 19:16

3.0.0

leaderElection

0

1

fast leader election test failing .

48010

No Perforce job exists for this issue.

4

33205

08/Oct/08 7:05 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061ov:

ZooKeeper

ZOOKEEPER-177

needed: docs for JMX

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

06/Oct/08 12:02

13/Feb/09 16:18

27/Jan/09 19:42

3.1.0

documentation, jmx

0

I don't know if this should be part of the logging section in the zookeeper admin guide (see JIRA ZOOKEEPER-166 )
but, per Patrick, "related to security, ack, yea another doc, is the documentation of the JMX support that zk has"

43031

No Perforce job exists for this issue.

1

33568

04/Nov/08 8:03 PM

11 years, 9 weeks, 1 day ago

Reviewed

0|i063xj:

ZooKeeper

ZOOKEEPER-176

needed: docs for auth plugin developers

Improvement

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:58

06/Oct/08 12:05

documentation

0

Part of the interest in documentation related to security:
how to add different/new methods for integrating zk into existing auth mechanisms, etc...

214077

No Perforce job exists for this issue.

0

68501

11 years, 25 weeks, 3 days ago

0|i0c3fr:

ZooKeeper

ZOOKEEPER-175

needed: docs for ops - how to setup acls & authentication in the server

Improvement

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:57

26/Jun/15 18:02

documentation

0

1

Part of the interest in creating documentation related to security.

214076

No Perforce job exists for this issue.

0

68502

26/Jun/15 6:02 PM

4 years, 38 weeks, 6 days ago

0|i0c3fz:

ZooKeeper

ZOOKEEPER-174

ZOOKEEPER-168 Content needed: "Sync vs Async"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:53

documentation

0

14400

0%

Some time ago, Patrick suggested a subsection of the Basic Building Blocks section of the Programmer's Guide called "Sync vs. Async Operations." I don't know if that is still on the roadmap, but I put it here in case anyone wants to provide content for me to add to the Programmer's Guide.

0%

14400

214075

No Perforce job exists for this issue.

0

68503

11 years, 25 weeks, 3 days ago

0|i0c3g7:

ZooKeeper

ZOOKEEPER-173

ZOOKEEPER-168 Content needed: "Miscellaneous Operations"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:50

documentation

0

14400

0%

For lack of better term, I'm adding a section to the programmer's guide called "Miscellaneous Operations". This might eventually be broken out into sections of more descriptive terms. In short, these are COMMON zookeeper programming operations that do not come under the headings of connecting, reading, writing, or watching.

(I don't think we should document uncommon or seldom used API here.)

0%

14400

214074

No Perforce job exists for this issue.

0

68504

11 years, 25 weeks, 3 days ago

0|i0c3gf:

ZooKeeper

ZOOKEEPER-172

ZOOKEEPER-168 Content needed: "Handling Watches"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:46

documentation

0

14400

0%

We need the content which describes how client code should handle watches. Ideally this would have pseudo code describing any lead-up (for example resource allocation, etc), a few lines of sample code in Java, and a few parallel lines in C.

Any commentary would be highly appreciated -- anything about handling watches that would not be obvious to a new ZooKeeper developer.

0%

14400

214073

No Perforce job exists for this issue.

0

68505

11 years, 25 weeks, 3 days ago

0|i0c3gn:

ZooKeeper

ZOOKEEPER-171

ZOOKEEPER-168 Content needed: "Write Operations"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:39

documentation

0

14400

0%

We need the content which describes how to a client should write data to a zookeeper node. Ideally this would have pseudo code describing any lead-up (for example resource allocation, etc), a few lines of sample code in Java, and a few parallel lines in C.

Any commentary would be highly appreciated -- anything about writing data that would not be obvious to a new ZooKeeper developer.

0%

14400

214072

No Perforce job exists for this issue.

0

68506

11 years, 25 weeks, 3 days ago

0|i0c3gv:

ZooKeeper

ZOOKEEPER-170

ZOOKEEPER-168 Content needed: "Read Operations"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:37

06/Oct/08 11:38

documentation

0

14400

0%

We need the content which describes how to a client should read data from a zookeeper node. Ideally this would have pseudo code describing any lead-up (for example resource allocation, etc), a few lines of sample code in Java, and a few parallel lines in C.

Any commentary would be highly appreciated -- anything about reading data that would not be obvious to a new ZooKeeper developer.

0%

14400

214071

No Perforce job exists for this issue.

0

68507

11 years, 25 weeks, 3 days ago

0|i0c3h3:

ZooKeeper

ZOOKEEPER-169

ZOOKEEPER-168 Content needed: "Connecting to ZooKeeper"

Sub-task

Resolved

Major

Fixed

Michael Tanenblatt

Robbie Scott

06/Oct/08 11:35

06/Aug/17 01:01

06/Aug/17 00:50

3.6.0

documentation

0

2

14400

0%

We need the content which describes how to a client should connect to a zookeeper server. Ideally this would have pseudo code describing any lead-up (for example resource allocation, etc), a few lines of sample code in Java, and a few parallel lines in C.

Any commentary would be highly appreciated. Stuff like, "You can open an unlimited number of ZooKeeper connections because..." or "Opening too many connections to ZooKeeper is not advisable because..." -- in short, anything that is not obvious to the new ZooKeeper developer.

0%

14400

214070

No Perforce job exists for this issue.

0

68508

10/Mar/17 7:57 AM

2 years, 32 weeks, 4 days ago

0|i0c3hb:

ZooKeeper

ZOOKEEPER-168

Programmer's Guide "Basic Operations" section is missing content

Bug

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:24

08/Oct/08 14:04

3.0.0

documentation

0

ZOOKEEPER-169, ZOOKEEPER-170, ZOOKEEPER-171, ZOOKEEPER-172, ZOOKEEPER-173, ZOOKEEPER-174, ZOOKEEPER-179

There is a chapter in the ZooKeeper Programmer's Guide called: "Building Blocks: A Guide to ZooKeeper Operations" that currently contains no content. The idea behind this section is document all the basic things you can do with the ZooKeeper API -- not every little detail, or even every call in the API, but the basics, like: how to connect to a server, how to read a node, how to write to a node, how to set a watch, etc. I will create a subtask for every section in this chapter.

I don't mind writing up the these sections, if I can get the information, whether its' by phone, email, im, etc.

0%

86400

214069

No Perforce job exists for this issue.

0

33206

11 years, 25 weeks, 2 days ago

0|i061p3:

ZooKeeper

ZOOKEEPER-167

ZOOKEEPER-160 Content needed: "Troubleshooting"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 11:04

20/Oct/08 17:24

0

28800

0%

We need the content / information / knowledge for the "Troubleshooting" subsection of the ZooKeeper Administrator's Guide. This is information about what to do when an existing deployment of ZooKeeper has problems.

0%

28800

214068

No Perforce job exists for this issue.

0

68509

20/Oct/08 5:24 PM

11 years, 23 weeks, 3 days ago

0|i0c3hj:

ZooKeeper

ZOOKEEPER-166

ZOOKEEPER-160 Content needed: "Logging"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:18

11/Mar/09 21:00

documentation

0

28800

0%

Content/information needed on Logging, from the point of view of the ZooKeeper Administrator

0%

28800

214067

No Perforce job exists for this issue.

0

68510

11 years, 25 weeks, 3 days ago

0|i0c3hr:

ZooKeeper

ZOOKEEPER-165

ZOOKEEPER-160 Content needed: "Monitoring"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:14

09/Jul/10 11:36

documentation

0

28800

0%

Content/information needed on how to Monitor a zooKeeper deployment. I think we can talk about jmx, etc

0%

28800

214066

No Perforce job exists for this issue.

0

68511

09/Jul/10 11:36 AM

9 years, 37 weeks, 6 days ago

0|i0c3hz:

ZooKeeper

ZOOKEEPER-164

ZOOKEEPER-160 Content needed: "Administering"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:12

11/Mar/09 21:00

documentation

0

28800

0%

Content/Information needed on the day to day running, management, administering of a ZooKeeper deployment.

0%

28800

214065

No Perforce job exists for this issue.

0

68512

11 years, 25 weeks, 3 days ago

0|i0c3i7:

ZooKeeper

ZOOKEEPER-163

ZOOKEEPER-160 Content needed: "Things to Consider: Strengths and Limitations"

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:11

documentation

0

I need content for the section "Things to Consider: ZooKeeper Strengths and Limitations" in the ZooKeeper admin guide.

214064

No Perforce job exists for this issue.

0

68513

11 years, 25 weeks, 3 days ago

0|i0c3if:

ZooKeeper

ZOOKEEPER-162

ZOOKEEPER-160 Content needed: Provisioning

Sub-task

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:09

03/Aug/11 04:53

documentation

0

28800

0%

I need the content for the ZooKeeper Admin guide, for the section on "Provisioning" your zookeeper deployment.

0%

28800

66419

No Perforce job exists for this issue.

0

68514

11 years, 25 weeks, 3 days ago

0|i0c3in:

ZooKeeper

ZOOKEEPER-161

ZOOKEEPER-160 Content needed: "Designing a ZooKeeper Deployment"

Sub-task

Closed

Major

Fixed

Benjamin Reed

Robbie Scott

06/Oct/08 10:05

13/Feb/09 16:18

09/Dec/08 19:12

3.0.1

3.1.0

documentation

0

28800

0%

We should document how ops engineer should design their ZooKeeper deployments.

0%

28800

48011

No Perforce job exists for this issue.

2

33569

09/Dec/08 7:05 PM

11 years, 16 weeks, 1 day ago

Reviewed

0|i063xr:

ZooKeeper

ZOOKEEPER-160

Admin / Ops Guide Missing Content

Bug

Open

Major

Unresolved

Unassigned

Robbie Scott

06/Oct/08 10:01

07/Oct/08 20:07

documentation

0

ZOOKEEPER-161, ZOOKEEPER-162, ZOOKEEPER-163, ZOOKEEPER-164, ZOOKEEPER-165, ZOOKEEPER-166, ZOOKEEPER-167, ZOOKEEPER-180

I'm adding new sections to the ZooKeeper Admin guide, but I don't have the content to fill them out. These sections are:
- Designing a ZooKeeper Deployment
- Provisioning
- Things to Consider: ZooKeeper Strengths and Limitations
- Administering
- Monitoring
- Logging
- Troubleshooting

I'll add a subtask for each section.

Note: If the area experts prefer, I can conduct a phone interview or email conversation or IM with them on the topic, and then write up the section so they don't have to.

0%

172800

214063

No Perforce job exists for this issue.

0

33207

11 years, 25 weeks, 2 days ago

0|i061pb:

ZooKeeper

ZOOKEEPER-159

Cover two corner cases of leader election

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

06/Oct/08 09:15

25/Oct/08 21:10

08/Oct/08 14:00

3.0.0

leaderElection

0

1

In the default implementation of leader election, there are two undesirable cases that need to be covered:

1- If there is a leader elected and this leader is supported by at least quorum of peers, then it can happen that one peer disconnects from the leader, and initiates a new leader election. As it is a new leader election, we increment the logical clock of this peer, and according to the current implementation, this peer won't accept any vote from a peer with a lower LE turn (corresponds to the value of the logical clock of the voting peer). The attached patch corrects this problem by allowing a peer to go back to a previous epoch in the case a majority votes for a leader, and the peer also receives a vote from the leader. This feature allows a peer to correct a false suspicion of the current leader;

2- If a peer advances to a new turn before others, then it may end up voting for a peer that either does not have the highest server id or the the highest zxid. The attached patch fixes this problem by resetting the vote a peer when it updates its logical clock upon receiving a notification with a higher turn value.

48012

No Perforce job exists for this issue.

1

33208

08/Oct/08 2:55 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061pj:

ZooKeeper

ZOOKEEPER-158

Leader and followers increase cpu utilization upon loss of a follower

Improvement

Closed

Minor

Fixed

Mahadev Konar

Flavio Paiva Junqueira

05/Oct/08 07:17

26/Mar/10 13:24

16/Nov/09 16:00

3.3.0

quorum

0

1

In a set of ZooKeeper servers, when there is a leader operation and supported by a quorum of servers, we have observed that cpu utilization increases substantially once a follower fails or disconnects. Stu Hood provided logs showing this behavior.

48013

No Perforce job exists for this issue.

1

33570

23/Oct/08 7:08 PM

10 years, 19 weeks, 3 days ago

0|i063xz:

ZooKeeper

ZOOKEEPER-157

Peer can't find existing leader

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

04/Oct/08 18:20

25/Oct/08 21:10

05/Oct/08 07:11

3.0.0

0

1

In the patch of JIRA 127, I forgot to set the state of a peer when this peer is looking for a leader and it receives a message from the current leader. In this patch, I have fixed this problem, and also returned to what we had previously. With this current patch, when a peer joins and there is already a leader elected, the joining peer will only recognize the new leader as the leader once it receives a confirmation from a majority. The alternative is to set the leader once we receive a message from a peer claiming to be the leader (what we have on trunk now, although broken because we don't set the state of the peer), but there could be cases in which a peer believes to be leader, although it is not the leader any longer, and the joining peer would select this false leader to be its leader. Eventually, the false leader would timeout, and both processes would select the correct leader. This small fix gets rid of such problems, though.

48014

No Perforce job exists for this issue.

2

33209

04/Oct/08 7:25 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061pr:

ZooKeeper

ZOOKEEPER-156

update programmer guide with acl details from old wiki page

Task

Closed

Major

Fixed

Patrick D. Hunt

02/Oct/08 18:39

25/Oct/08 21:10

03/Oct/08 19:35

3.0.0

documentation

0

Need to update the programmer guide with the acl details form the old wiki page.

http://zookeeper.wiki.sourceforge.net/ZooKeeperACLs

48015

No Perforce job exists for this issue.

3

33571

03/Oct/08 6:55 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063y7:

ZooKeeper

ZOOKEEPER-155

improve "the zookeeper project" section of overview doc

Bug

Closed

Minor

Fixed

Patrick D. Hunt

02/Oct/08 09:28

25/Oct/08 21:10

03/Oct/08 19:26

3.0.0

documentation

0

1

ZOOKEEPER-154

In "the zookeeper project" section of
http://hadoop.apache.org/zookeeper/docs/current/zookeeperOver.html

we should remove the hod reference (doesn't use zk) and also update the second paragraph, perhaps remove? Since we are already on the apache zk page.

48016

No Perforce job exists for this issue.

0

33210

04/Oct/08 7:17 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061pz:

ZooKeeper

ZOOKEEPER-154

reliability graph diagram in overview doc needs context

Bug

Closed

Minor

Fixed

Patrick D. Hunt

02/Oct/08 09:20

25/Oct/08 21:10

03/Oct/08 19:27

3.0.0

documentation

0

1

ZOOKEEPER-155

from question on the user mailing list:

the "Reliability in the Presence of Errors" graph on http://hadoop.apache.org/zookeeper/docs/current/zookeeperOver.html does not list how many ZooKeeper quorum nodes are in use, or the fraction of reads/writes.

ben mentioned:

Here is the missing text:
To show the behavior of the system over time as failures are injected we
ran a ZooKeeper service made up of 7 machines. We ran the same
saturation benchmark as before, but this time we kept the write
percentage at a constant 30\%, which is a conservative ratio of our
expected workloads.

48017

No Perforce job exists for this issue.

3

33211

03/Oct/08 6:34 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061q7:

ZooKeeper

ZOOKEEPER-153

add api support for "subscribe" method

New Feature

Open

Minor

Unresolved

Unassigned

Patrick D. Hunt

01/Oct/08 18:42

16/Nov/19 03:02

c client, documentation, java client, server, tests

6

14

Subscribe Method
(note, this was moved from http://zookeeper.wiki.sourceforge.net/SubscribeMethod)

Outline of the semantics and the requirements of a yet-to-be-implemented subscribe() method.

Background

ZooKeeper uses a very light weight one-time notification method for notifying interested clients of changes to ZooKeeper data nodes (znode). Clients can set a watch on a node when they request information about a znode. The watch is atomically set and the data returned, so that any subsequent changes to the znode that affect the data returned will trigger a watch event. The watch stays in place until triggered or the client is disconnected from a ZooKeeper server. A disconnect watch event implicitly triggers all watches.

ZooKeeper users have wondered if they can set permanent watches rather than one time watches. In reality such permanent watches do not provide any extra benefit over one time watches. Specifically, no data is included in a watch event, so the client still needs to do a query operation to get the data corresponding to a change; even then, the znode can change yet again after the event is received and before the client sends the query operation. Even the number of of changes to a znode can be found using one time watches and checking the mzxid in the stat structure of the znode. And the client will still miss events that happen when the client switches ZooKeeper servers.

There are use cases that require clients to see every change to a ZooKeeper node. The most general case is when a client behaves like a state machine and each change to the znode changes the state of the client. In these cases ZooKeeper is much more like a publish/subscribe system than a distributed register. To support this case we need not only reliable permanent watches (we even get the events that happen while switching servers) but also the data that caused the change, so that the client doesn't miss data that occurs between rapid fire changes.

Semantics

The subscribe(String path) causes ZooKeeper to register a subscription for a znode. The initial value of the znode and any subsequent changes to that znode will cause a watch event with the data to be sent to the client. The client will see all changes in order. If a client switches servers, any missed events with the corresponding data will be sent to the client when the client reconnects to a server.

There are three ways to cancel a subscription:

1. Calling unsubscribe(String path)
2. Closing the ZooKeeper session or letting it expire
3. Falling too far behind. If the server decides that a client is not processing the watch events fast enough, it will cancel the subscription and send a SUBSCRIPTION_CANCELLED watch event.

Requirements

There are a couple of things that make it hard to implement the subscribe() method:

1. Servers must have complete transaction logs - Currently ZooKeeper servers just need to have their data trees and in flight transaction logs in sync. When a follower syncs to a leader, the leader can just blast down a new snapshot of its data tree; it does not need to send past transactions that the follower might have missed. However in order to send changes that might have been missed by a client, the ZooKeeper server must be able to look into the past to send missed changes.
2. Servers must be able to send clients information about past changes - Currenly ZooKeeper servers just send clients information about the current state of the system. However, to implement subscribe clients must be able to go back into the log and send watches for past changes.

Implementation Hints

There are things that work in our favor. ZooKeeper does have a bound on the amount of time it needs to look into the past. A ZooKeeper server bounds the session expiration time. The server does not need to keep a record of transactions older than this bound.

ZooKeeper also keeps a log of transactions. As long as the log is complete enough (as all the transaction back to the longest expiration time) the server has the information it needs and it isn't hard to process.

We do not want to cause the log disk to seek while looking at past transactions. There are two complimentary approaches to handling this problems: keep a few of the transactions from the recent past in memory and log to two disks. The first log disk will be synced before letting requests proceed and the second disk will not be synced. Recovery uses the first log disk and ensures that the second log disk has the same log at recovery time. The second log disk is to look into the past. Using the two disks in this way allows synchronous logging to be fast because seeks are avoided on the disk with the synchronous log.

214062

No Perforce job exists for this issue.

0

68515

15/Nov/19 8:05 PM

17 weeks, 5 days ago

0|i0c3iv:

ZooKeeper

ZOOKEEPER-152

Improve unit tests for leader election

Improvement

Closed

Minor

Fixed

Flavio Paiva Junqueira

01/Oct/08 16:29

26/Mar/10 13:24

25/Jan/10 19:10

3.3.0

quorum

0

1

ZOOKEEPER-658

There are two possible tasks here:

1- Change the algorithm tested on QuorumTest.java;
2- Add tests for the other supported algorithms.

48018

No Perforce job exists for this issue.

0

33572

15/Jan/10 5:05 PM

10 years, 9 weeks, 3 days ago

Reviewed

0|i063yf:

ZooKeeper

ZOOKEEPER-151

Document change to server configuration

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

01/Oct/08 16:23

25/Oct/08 21:10

16/Oct/08 17:55

3.0.0

0

1

The patch of jira 127 changed the format of server configuration files, but it didn't change the documentation.

48019

No Perforce job exists for this issue.

3

33212

15/Oct/08 7:16 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061qf:

ZooKeeper

ZOOKEEPER-150

zookeeper build broken

Bug

Closed

Blocker

Fixed

Mahadev Konar

01/Oct/08 13:28

25/Oct/08 21:10

01/Oct/08 15:06

3.0.0

0

1

the build is broekn with ZOOKEEPER-38 checked in.

48020

No Perforce job exists for this issue.

1

33213

01/Oct/08 3:06 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061qn:

ZooKeeper

ZOOKEEPER-149

c interface is missing tests against java server (mock only)

Improvement

Closed

Major

Duplicate

Benjamin Reed

Patrick D. Hunt

30/Sep/08 22:51

13/Feb/09 16:18

27/Jan/09 19:52

3.1.0

c client

0

ZOOKEEPER-23, ZOOKEEPER-215

The c client interface has unit tests but they are against mock server implementations only. We need to add tests for the c interface against live java server.

214061

No Perforce job exists for this issue.

0

33573

20/Oct/08 8:32 AM

11 years, 11 weeks, 1 day ago

0|i063yn:

ZooKeeper

ZOOKEEPER-148

add jdiff support to zookeeper release process

Improvement

Closed

Minor

Fixed

Mahadev Konar

Patrick D. Hunt

26/Sep/08 16:16

08/Jul/09 16:23

15/May/09 13:03

3.2.0

build

0

1

HADOOP-3532, ZOOKEEPER-371, HADOOP-2421

Add jdiff support to enable detailed release notes on changes to java.

Hadoop just added code for including that in the build: HADOOP-2421.

48021

No Perforce job exists for this issue.

0

33574

24/Apr/09 3:16 PM

10 years, 45 weeks, 6 days ago

0|i063yv:

ZooKeeper

ZOOKEEPER-147

This is version of the documents with most of the [tbd...] scrubbed out

Improvement

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

25/Sep/08 23:50

25/Oct/08 21:10

01/Oct/08 18:29

3.0.0

documentation

0

345600

0%

The docs had lots of comments to myself marked [tbd..]. I scrubbed these out. I still have a list I will work from, but the [tbd]s are probably sensible to anyone else. For a few cases where it's obvious what a tbd is asking, I left in in, just in case someone else wants to take a crack at fixing them, knows the answer, etc.

The estimate below refers to how many full days I think it would take to get all the tbd's answered and put back into the document.

0%

345600

48022

No Perforce job exists for this issue.

2

33575

30/Sep/08 10:01 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063z3:

ZooKeeper

ZOOKEEPER-146

Allow clients to retrieve the list of hosts/servers from a URL

Improvement

Open

Major

Unresolved

Unassigned

Jakob Homan

25/Sep/08 19:10

14/Dec/19 06:08

3.0.0

3.7.0

java client

0

2

ZOOKEEPER-836, ZOOKEEPER-237, ZOOKEEPER-390

As I suggested in ZOOKEEPER-107, it would be nice to allow clients to specify their hosts strings via a URL so that this can change dynamically. For instance, obtain the current list of hosts from a webpage or a file on disk.

70776

No Perforce job exists for this issue.

1

68516

26/Sep/08 4:36 PM

10 years, 47 weeks, 2 days ago

not a blocker for 3.2, moving to 3.3

0|i0c3j3:

ZooKeeper

ZOOKEEPER-145

write detailed release notes for users migrating from 2.x to 3.0

Task

Closed

Critical

Fixed

Patrick D. Hunt

25/Sep/08 18:43

25/Oct/08 21:10

21/Oct/08 21:26

3.0.0

c client, documentation, java client

0

ZOOKEEPER-186

We need to write detailed release notes that detail how c and java client API users as well as zk server administrators migrate from 2.x to 3.x zookeeper release.

48023

No Perforce job exists for this issue.

1

33576

26/Sep/08 2:47 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063zb:

ZooKeeper

ZOOKEEPER-144

add tostring support for watcher event, and enums for event type/state

Improvement

Closed

Minor

Fixed

Jakob Homan

Patrick D. Hunt

25/Sep/08 18:29

25/Oct/08 21:10

30/Sep/08 22:48

3.0.0

java client

0

ZOOKEEPER-139, ZOOKEEPER-114

it would be great if we had toString capability on watcher event, and enums for type/state. the logs would certainly benefit (when fixing this try to update any places where we are logging event information)

48024

No Perforce job exists for this issue.

1

33577

30/Sep/08 3:50 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063zj:

ZooKeeper

ZOOKEEPER-143

Exceeding max data node size causes disconnect

Improvement

Open

Minor

Unresolved

Unassigned

Patrick D. Hunt

25/Sep/08 17:38

25/Sep/08 17:39

server

0

ZOOKEEPER-142

The server will disconnect the client if the max node size is exceeded. See ZOOKEEPER-142

We should consider whether disconnect is warranted (142 has added documentation for this to the java client)
We should also document this in the main forrest based docs.

214060

No Perforce job exists for this issue.

0

68517

11 years, 27 weeks ago

0|i0c3jb:

ZooKeeper

ZOOKEEPER-142

Provide Javadoc as to the maximum size of the data byte array that may be stored within a znode

Improvement

Closed

Trivial

Fixed

Jakob Homan

24/Sep/08 22:08

25/Oct/08 21:10

25/Sep/08 17:38

3.0.0

java client

0

ZOOKEEPER-143

Currently there is nothing in the API documentation as to the maximum size of the data in a znode.

48025

No Perforce job exists for this issue.

1

33578

25/Sep/08 5:34 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i063zr:

ZooKeeper

ZOOKEEPER-141

Remove challenge on QuorumCnxManager

Improvement

Closed

Minor

Fixed

Unassigned

Flavio Paiva Junqueira

15/Sep/08 13:27

26/Mar/10 13:30

12/Nov/09 10:25

0

ZOOKEEPER-127

The patch of JIRA 127 replaces InetSocket objects with server ids on QuorumCnxManager. Server ids are unique by assumption, and can be used instead of the randomly generated challenges we have currently. I suggest that we make this modification, and we use server ids as the challenge values instead of randomly generated numbers.

48026

No Perforce job exists for this issue.

0

33579

10 years, 20 weeks ago

0|i063zz:

ZooKeeper

ZOOKEEPER-140

Deadlock in QuorumCnxManager

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

13/Sep/08 06:02

25/Oct/08 21:10

01/Oct/08 05:21

3.0.0

1

2

ZOOKEEPER-127

Frequently the servers deadlock in QuorumCnxManager:initiateConnection on
s.read(msgBuffer) when reading the challenge from the peer.

Calls to initiateConnection and receiveConnection are synchronized, so only one or the other can be executing at a time. This prevents two connections from opening between the same pair of servers.

However, it seems that this leads to deadlock, as in this scenario:

{noformat}
A (initiate --> B)
B (initiate --> C)
C (initiate --> A)
{noformat}

initiateConnection can only complete when receiveConnection runs on the remote peer and answers the challenge. If all servers are blocked in initiateConnection, receiveConnection never runs and leader election halts.

48027

No Perforce job exists for this issue.

0

33214

16/Sep/08 7:31 AM

11 years, 22 weeks, 4 days ago

0|i061qv:

ZooKeeper

ZOOKEEPER-139

ZOOKEEPER-7 Create Enums for WatcherEvent's KeeperState and EventType

Sub-task

Closed

Major

Fixed

Jakob Homan

10/Sep/08 22:36

25/Oct/08 21:10

25/Sep/08 18:25

3.0.0

0

ZOOKEEPER-18, ZOOKEEPER-144

48028

No Perforce job exists for this issue.

3

33580

15/Sep/08 7:49 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i06407:

ZooKeeper

ZOOKEEPER-138

c client watcher objects can lose events

Bug

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

08/Sep/08 17:03

25/Oct/08 21:10

21/Oct/08 21:23

3.0.0

c client

0

1

ZOOKEEPER-23

if a single watcher (A) is registered on a single node for both a getdata and exists watch the second watch event may be lost:

1) getdata("node", A)
2) setdata("node"...)
3) exists("node", A)
4) delete("node"...)

if watch events for 2 is processed on the client (zookeeper.java, zkwatcher) after 3 completes then the zkwatcher process event method will clear the watch and the subsequent operation's (4) event will be ignored

48029

No Perforce job exists for this issue.

0

33215

21/Oct/08 2:13 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061r3:

ZooKeeper

ZOOKEEPER-137

client watcher objects can lose events

Bug

Closed

Major

Fixed

Patrick D. Hunt

08/Sep/08 17:02

25/Oct/08 21:10

24/Sep/08 17:05

3.0.0

java client

0

1

ZOOKEEPER-124

if a single watcher (A) is registered on a single node for both a getdata and exists watch the second watch event may be lost:

1) getdata("node", A)
2) setdata("node"...)
3) exists("node", A)
4) delete("node"...)

if watch events for 2 is processed on the client (zookeeper.java, zkwatcher) after 3 completes then the zkwatcher process event method will clear the watch and the subsequent operation's (4) event will be ignored

48030

No Perforce job exists for this issue.

1

33216

24/Sep/08 11:38 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061rb:

ZooKeeper

ZOOKEEPER-136

sync causes hang in all followers of quorum

Bug

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

05/Sep/08 14:27

25/Oct/08 21:10

03/Oct/08 09:12

3.0.0

0

1

The attached test causes all of the followers of a quorum to hang. Leader continues to function correctly.

48031

No Perforce job exists for this issue.

1

33217

07/Sep/08 11:51 AM

11 years, 22 weeks, 4 days ago

0|i061rj:

ZooKeeper

ZOOKEEPER-135

Fat jar build target

New Feature

Closed

Minor

Fixed

Patrick D. Hunt

Benjamin Reed

04/Sep/08 11:58

13/Feb/09 16:18

08/Jan/09 17:32

3.1.0

build

0

For testing and experimentation purposes it would be nice to have everything in a self contained executable jar file that you can plop down on a machine and run.

48032

No Perforce job exists for this issue.

5

33581

16/Sep/08 8:21 PM

11 years, 11 weeks, 2 days ago

Reviewed

0|i0640f:

ZooKeeper

ZOOKEEPER-134

findbugs cleanup

Improvement

Closed

Major

Fixed

Patrick D. Hunt

03/Sep/08 19:06

25/Oct/08 21:10

04/Sep/08 13:02

3.0.0

0

cleaned up a number of findbugs warnings, in particular:

1) added @overrides appropriately
2) import cleanups
3) log or document many empty blocks
4) resource cleanup (usually by adding finally blocks)

48033

No Perforce job exists for this issue.

1

33582

04/Sep/08 11:49 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0640n:

ZooKeeper

ZOOKEEPER-133

hudson tests failing intermittently

Bug

Closed

Major

Fixed

Patrick D. Hunt

03/Sep/08 16:46

25/Oct/08 21:10

05/Sep/08 10:39

3.0.0

tests

0

1

There is a bug in the ClientTest.java unit test, timing issue in "withWatcherObj" test. Patch forthcoming.

48034

No Perforce job exists for this issue.

2

33218

04/Sep/08 11:39 AM

11 years, 22 weeks, 4 days ago

0|i061rr:

ZooKeeper

ZOOKEEPER-132

ZOOKEEPER-7 Create Enum to replace CreateFlag in ZooKepper.create method

Sub-task

Closed

Major

Fixed

Jakob Homan

02/Sep/08 23:20

25/Oct/08 21:10

08/Sep/08 12:56

3.0.0

java client

0

Replace the current named constant integer in the craete method with a typesafe enum value.

48035

No Perforce job exists for this issue.

3

33583

03/Sep/08 5:18 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0640v:

ZooKeeper

ZOOKEEPER-131

Old leader election can elect a dead leader over and over again

Bug

Closed

Major

Fixed

Benjamin Reed

02/Sep/08 13:27

25/Oct/08 21:10

18/Sep/08 21:25

3.0.0

leaderElection

0

2

I think there is a race condition that is probably easy to get into with the old leader election and a large number of servers:

1) Leader dies
2) Followers start looking for a new leader before all Followers have abandoned the Leader
3) The Followers looking for a new leader see votes of Followers still following the (now dead) Leader and start voting for the dead Leader
4) The dead Leader gets reelected.

For the old leader election a server should not vote for another server that is not nominating himself.

48036

No Perforce job exists for this issue.

3

33219

17/Sep/08 7:41 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061rz:

ZooKeeper

ZOOKEEPER-130

update build.xml to support apache release process

Improvement

Closed

Major

Fixed

Patrick D. Hunt

29/Aug/08 19:33

25/Oct/08 21:10

10/Sep/08 13:49

3.0.0

build

0

1

update the build process to support "tar" and "binary" targets similar in structure to hadoop-core build. this is needed as part of the 3.0.0 release process.

48037

No Perforce job exists for this issue.

2

33584

09/Sep/08 11:55 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i06413:

ZooKeeper

ZOOKEEPER-129

Thrift proxy

New Feature

Open

Trivial

Unresolved

Unassigned

Chris Smith

29/Aug/08 13:11

29/Aug/08 17:01

0

2

This is a basic server that acts as a proxy between a thrift interface and ZooKeeper. It doesn't support watches, and I'm not entirely convinced the session handling makes sense, but it works for basic operations like creating, listing and deleting nodes. The patch puts the code and (basic) build file under src/contrib/thrift; the build file requires the thrift compiler to be in the path, and the thrift and zookeeper libraries in the lib dir.

214059

No Perforce job exists for this issue.

1

68518

29/Aug/08 2:18 PM

11 years, 30 weeks, 6 days ago

0|i0c3jj:

ZooKeeper

ZOOKEEPER-128

test coverage on async client operations needs to be improved

Improvement

Closed

Major

Fixed

Patrick D. Hunt

28/Aug/08 15:42

25/Oct/08 21:10

05/Sep/08 16:21

3.0.0

java client

0

clover reports limited testing of the async client operations. need addition tests to verify this functionality.

48038

No Perforce job exists for this issue.

2

33585

04/Sep/08 11:31 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0641b:

ZooKeeper

ZOOKEEPER-127

Use of non-standard election ports in config breaks services

Bug

Closed

Critical

Fixed

Flavio Paiva Junqueira

Mark Harwood

27/Aug/08 09:18

12/Nov/09 10:23

01/Oct/08 04:40

3.0.0

quorum

0

3

ZOOKEEPER-140, ZOOKEEPER-141

In QuorumCnxManager.toSend there is a call to create a connection as follows:
channel = SocketChannel.open(new InetSocketAddress(addr, port));

Unfortunately "addr" is the ip address of a remote server while "port" is the electionPort of *this* server.
As an example, given this configuration (taken from my zoo.cfg)
server.1=10.20.9.254:2881
server.2=10.20.9.9:2882
server.3=10.20.9.254:2883
Server 3 was observed trying to make a connection to host 10.20.9.9 on port 2883 and obviously failing.

In tests where all machines use the same electionPort this bug would not manifest itself.

48039

No Perforce job exists for this issue.

7

33220

27/Aug/08 9:25 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061s7:

ZooKeeper

ZOOKEEPER-126

zookeeper client close operation may block indefinitely

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

26/Aug/08 17:25

14/Dec/19 06:09

3.7.0

java client

3

4

ZOOKEEPER-846, ZOOKEEPER-63, ZOOKEEPER-1202

Moving the hang issue from ZOOKEEPER-63 to here. See 63 for background and potential patch (patch_ZOOKEEPER-63.patch).

specifically (from James):

"I'm thinking the close() method should not wait() forever on the disconnect packet, just a closeTimeout length - say a few seconds. Afterall blocking and forcing a reconnect just to redeliver the disconnect packet seems a bit silly - when the server has to deal with clients which just have their sockets fail anyway"

2179

No Perforce job exists for this issue.

2

33221

23/Oct/08 7:16 PM

8 years, 11 weeks, 2 days ago

0|i061sf:

ZooKeeper

ZOOKEEPER-125

Class not needed on FastLeaderElection

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

26/Aug/08 07:03

25/Oct/08 21:10

27/Aug/08 01:12

3.0.0

0

There is a class (ElectionResult) on both FastLeaderElection and AuthFastLeaderElection that was needed originally in LeaderElection, and it was copied and pasted unnecessarily, but it is not necessary in those classes.

48040

No Perforce job exists for this issue.

1

33586

27/Aug/08 1:11 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0641j:

ZooKeeper

ZOOKEEPER-124

StatCallback is broken in trunk

Bug

Closed

Major

Invalid

Unassigned

Stu Hood

24/Aug/08 21:00

25/Oct/08 21:10

26/Sep/08 14:02

3.0.0

java client, server, tests

0

1

ZOOKEEPER-137

Linux corner-cube 2.6.24-19-generic #1 SMP Fri Jul 11 23:41:49 UTC 2008 i686 GNU/Linux
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing)

StatCallback appears to be broken in trunk. I'll attach a patch for AsyncTest that triggers the behaviour.

214058

No Perforce job exists for this issue.

2

33222

25/Aug/08 4:37 PM

11 years, 22 weeks, 4 days ago

0|i061sn:

ZooKeeper

ZOOKEEPER-123

In two places, the wrong class is specified for the logger

Bug

Closed

Minor

Fixed

Jakob Homan

24/Aug/08 15:29

25/Oct/08 21:10

25/Aug/08 14:17

3.0.0

java client, tests

0

1

Copy and paste strikes again. In two logger instantiations, the wrong class is passed to the constructor. In ClientTest.java and ClientCnxn.java

48041

No Perforce job exists for this issue.

1

33223

25/Aug/08 2:09 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061sv:

ZooKeeper

ZOOKEEPER-122

NPE in jute's Utils.toCSVString

Bug

Closed

Major

Fixed

Anthony Urso

18/Aug/08 20:05

25/Oct/08 21:10

19/Aug/08 14:01

3.0.0

0

1

java.lang.NullPointerException
at org.apache.jute.Utils.toCSVString(Utils.java:128)
at org.apache.jute.CsvOutputArchive.writeString(CsvOutputArchive.java:94
)
at org.apache.zookeeper.proto.WatcherEvent.toString(WatcherEvent.java:60
)
at java.lang.String.valueOf(String.java:2827)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at com.liveoffice.mailindex.watchers.SuicidalWatcher.process(SuicidalWat
cher.java:11)
at org.apache.zookeeper.ZooKeeper.processWatchEvent(ZooKeeper.java:157)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:268)

48042

No Perforce job exists for this issue.

1

33224

19/Aug/08 1:37 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061t3:

ZooKeeper

ZOOKEEPER-121

SyncRequestProcessor is not closing log stream during shutdown

Bug

Closed

Blocker

Fixed

Mahadev Konar

Patrick D. Hunt

12/Aug/08 14:57

24/Jun/11 00:53

16/Mar/10 18:39

3.3.0

server

0

2

ZOOKEEPER-1109

The SyncRequestProcessor is not closing log stream during shutdown.

See FIXMEs with this ID in ClientBase.java -- I've commented out the assertion for the time being (checking for logs being deleted), as part of this fix re-enable these asserts and also verify tests on a Windows system.

48043

No Perforce job exists for this issue.

8

33225

09/Mar/10 2:50 AM

10 years, 2 weeks, 2 days ago

Reviewed

0|i061tb:

ZooKeeper

ZOOKEEPER-120

NIOServerCnxn needs to improve shutdown() handling

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

12/Aug/08 14:27

server

1

When the NIOServerCnxn is shutdown it doesn't handle gracefully - if shutdown is called we should set internal state. In the run method we print unnecessary error messages as we don't currently know that shutdown (gracefully) was called rather than some other more serious, unexpected, condition.

214057

No Perforce job exists for this issue.

0

33226

11 years, 33 weeks, 2 days ago

0|i061tj:

ZooKeeper

ZOOKEEPER-119

Reserve ".zookeeper" node for server use.

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

11/Aug/08 14:10

14/Jun/10 19:43

server

0

2

ZOOKEEPER-231

The server should not allow clients to create .zookeeper nodes in the node hierarchy. These nodes should be reserved for zk future use, some ideas:

* /.zookeeper/proc
* /.zookeeper/stats
* /.zookeeper/...

* /.../.zookeeper/... (disallow both at root as well as child nodes)

214056

No Perforce job exists for this issue.

1

68519

12/Aug/08 6:20 PM

9 years, 41 weeks, 3 days ago

0|i0c3jr:

ZooKeeper

ZOOKEEPER-118

findbugs flagged switch statement in followerrequestprocessor.run()

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

11/Aug/08 14:07

25/Oct/08 21:10

09/Sep/08 13:56

3.0.0

server

0

1

followerrequestprocessor:

is the case statement for SYNC supposed to fall through?
switch (request.type) {
case OpCode.sync:
zks.pendingSyncs.add(request);
case OpCode.create:

Please update the docs/code appropriately (if correct note it with comment just after the sync case statement.

48044

No Perforce job exists for this issue.

1

33227

08/Sep/08 4:52 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061tr:

ZooKeeper

ZOOKEEPER-117

threading issues in Leader election

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

11/Aug/08 14:04

25/Oct/08 21:10

24/Sep/08 17:28

3.0.0

server

0

2

Leader.lead() creates a new thread that can never terminate (short of restarting vm)

naked notifyall in lead() method - what is the condition variable? Best if set inside the sync block

48045

No Perforce job exists for this issue.

2

33228

09/Sep/08 11:11 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061tz:

ZooKeeper

ZOOKEEPER-116

simplify testing with quorums - quorum setup test utility code

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

08/Aug/08 02:25

tests

0

2

pull out the quorum starting and stopping code from the quorum servers and create a utility class that allows creation of n number of quorum servers in the tests

214055

No Perforce job exists for this issue.

0

68520

11 years, 33 weeks, 6 days ago

0|i0c3jz:

ZooKeeper

ZOOKEEPER-115

Potential NPE in QuorumCnxManager

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

07/Aug/08 17:10

25/Oct/08 21:10

13/Oct/08 10:11

3.0.0

quorum

0

2

Findbugs flagged this, notice that we are checking for null after using the reference.

if (senderWorkerMap.get(s.socket().getInetAddress()) != null) {
senderWorkerMap.get(s.socket().getInetAddress()).finish();
}

/*
* Start new worker thread with a clean state.
*/
if (s != null) {

48046

No Perforce job exists for this issue.

1

33229

19/Aug/08 8:24 AM

11 years, 22 weeks, 4 days ago

0|i061u7:

ZooKeeper

ZOOKEEPER-114

cleanup ugly event messages in zookeeper client

Improvement

Closed

Trivial

Fixed

Jakob Homan

Patrick D. Hunt

06/Aug/08 19:31

25/Oct/08 21:10

30/Sep/08 22:49

3.0.0

java client

0

ZOOKEEPER-144

Cleanup the event messages output by the java command line client:

The java client ZooKeeper.java has a process method in MyWatcher that prints raw event information to the console. Many new users find this confusing (esp since state changes have a null path).

public void process(WatcherEvent event) {
System.err.println(event.getPath() + ": " + event.getState() + "-"
+ event.getType());
}

If this is a state change we should print that, if it's an event we should print that (state change doesn't have a path). Somewhere in the message we should also indicate that the client received a watch notification from the server.

48047

No Perforce job exists for this issue.

0

33587

30/Sep/08 4:03 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0641r:

ZooKeeper

ZOOKEEPER-113

ZooKeeper.java should be interface not concrete class.

Improvement

Resolved

Major

Invalid

Unassigned

Patrick D. Hunt

06/Aug/08 17:05

25/Nov/08 18:56

java client

0

1

I think ZooKeeper class in src/java/main should be an interface rather than a concrete class. Among other OO goodness this would give us more flexibility when implementing tests on client code. Would also require something like a factory to be created, which might actually work well with another JIRA we have which is the idea to use a URI(s) rather than a host:port combination when creating zookeeper clients.

Unfortunately this would have a big impact on clients as it's not b/w compatible (instantiating a new client that is).

214054

No Perforce job exists for this issue.

0

33588

07/Aug/08 10:18 AM

11 years, 18 weeks, 2 days ago

Incompatible change

0|i0641z:

ZooKeeper

ZOOKEEPER-112

src/java/main ZooKeeper.java has test code embedded into it.

Bug

Closed

Major

Fixed

Patrick D. Hunt

06/Aug/08 17:01

25/Oct/08 21:10

05/Sep/08 16:31

3.0.0

tests

0

1

src/java/main ZooKeeper.java has a method "public void disconnect()" that is not part of the public api but put there for testing purposes (to test disconnection of the client from the server w/o actually shutting down the session)

This method needs to be moved out of the public api. preferably we should have a subclass in the test code itself that provides this method.

48048

No Perforce job exists for this issue.

3

33230

04/Sep/08 11:01 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061uf:

ZooKeeper

ZOOKEEPER-111

significant cleanup of existing tests

Improvement

Closed

Major

Fixed

Patrick D. Hunt

06/Aug/08 15:08

12/Jun/18 00:43

15/Aug/08 18:03

3.0.0

tests

0

ZOOKEEPER-2421, ZOOKEEPER-54, ZOOKEEPER-86, ZOOKEEPER-61

About to submit a patch that significantly cleans up existing tests.

1) removed the need for "sleep" calls in the tests, instead I monitor the status of the server (using the client socket "stat" command) which provides a barrier to the client test operations. I use this barrier both when starting and ending the test.

2) also fixed a number of problems in the tests where the test itself was broken.

3) general cleanup and some refactoring to make it easier to write new tests, maintain old, and track down issues if the test does find a problem.

48049

No Perforce job exists for this issue.

8

33589

07/Aug/08 5:17 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i06427:

ZooKeeper

ZOOKEEPER-110

Build script relies on svnant, which is not compatible with subversion 1.5 working copies

Bug

Closed

Major

Fixed

Jakob Homan

05/Aug/08 14:18

25/Oct/08 21:10

15/Aug/08 14:01

3.0.0

build

0

3

Subversion 1.5 command line, or subclipse version 1.4.x

The current build.xml ant script uses svnant to obtain the latest revision number from the repo, however svnant is not compatible with subversion 1.5 (http://subversion.tigris.org/svn_1.5_releasenotes.html), and so the build fails with working copies checked out by this version. The build fails with "this version of subversion is too old, please get a newer version..." This will become more apparent as svn 1.5 trickles out; I'm using a brand new dev environment with both subclipse 1.4 and svn 1.5 client, so I got bit rather quickly.

Those with svn 1.5 can get the code from the trunk, but cannot do an ant build.

svnant hasn't been updated in more than a year and appears to be dead, so it may no longer be a viable tool for the ant build.

48050

No Perforce job exists for this issue.

4

33231

05/Aug/08 4:49 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061un:

ZooKeeper

ZOOKEEPER-109

cleanup of NPE and Resource issue nits found by static analysis

Bug

Closed

Major

Fixed

Patrick D. Hunt

01/Aug/08 18:01

25/Oct/08 21:10

08/Aug/08 15:14

3.0.0

server

1

NPE/ResourceLeak cleanup for issues found during static analysis.

48051

No Perforce job exists for this issue.

1

33232

08/Aug/08 1:22 PM

11 years, 22 weeks, 4 days ago

0|i061uv:

ZooKeeper

ZOOKEEPER-108

sync implementation reorders operations

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

31/Jul/08 22:48

25/Oct/08 21:10

09/Aug/08 01:02

3.0.0

server

0

2

The current implementation of sync is broken. There is a race condition that causes a follower to return operations out of order, causing clients to drop their connections to a server.

I'll be attaching a patch to fix this problem shortly.

48052

No Perforce job exists for this issue.

4

33233

01/Aug/08 12:38 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061v3:

ZooKeeper

ZOOKEEPER-107

Allow dynamic changes to server cluster membership

Improvement

Resolved

Major

Fixed

Alexander Shraer

Patrick D. Hunt

29/Jul/08 17:02

13/Aug/18 07:36

29/Mar/13 11:30

3.5.0

server

28

80

ZOOKEEPER-762, ZOOKEEPER-1044, ZOOKEEPER-1113, ZOOKEEPER-1411

ZOOKEEPER-1660, ZOOKEEPER-1044, ZOOKEEPER-1411, ZOOKEEPER-1113, ZOOKEEPER-1549, ZOOKEEPER-1659, ZOOKEEPER-1561, ZOOKEEPER-1620, ZOOKEEPER-1625, ZOOKEEPER-2014, ZOOKEEPER-1633, ZOOKEEPER-1355, ZOOKEEPER-1045, ZOOKEEPER-368, ZOOKEEPER-390, ZOOKEEPER-1682, ZOOKEEPER-1324

Currently cluster membership is statically defined, adding/removing hosts to/from the server cluster dynamically needs to be supported.

60343

No Perforce job exists for this issue.

49

2349

31/Jul/08 6:29 PM

5 years, 15 weeks, 1 day ago

Reviewed

0|i00r9j:

ZooKeeper

ZOOKEEPER-106

CancelledKeyException

Bug

Open

Major

Unresolved

Unassigned

Flavio Paiva Junqueira

26/Jul/08 11:03

3

5

This exception seems to be harmless, but I thought it would be a good idea to document it just in case I overlooked anything. I think it is harmless because it is thrown inside the while loop that obtains the keys ready for I/O on Factory.run(). The penalty is skipping all other keys available in the for loop that follows the select call, but since we call again select in the next interation, we don't miss much. An optimizatio, however, would be catching this exception inside the for loop to avoid the extra call to select.

{noformat}
2008-07-24 21:09:03,894 - ERROR - [NIOServerCxn.Factory:NIOServerCnxn$Factory@152] - FIXMSG
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
at com.yahoo.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:136)
2008-07-24 21:09:06,912 - ERROR - [NIOServerCxn.Factory:NIOServerCnxn$Factory@152] - FIXMSG
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
at com.yahoo.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:136)
{noformat}

214053

No Perforce job exists for this issue.

0

33234

11 years, 35 weeks, 5 days ago

0|i061vb:

ZooKeeper

ZOOKEEPER-105

ZooKeeper java client main loop crashes on KeeperExceptions

Bug

Closed

Minor

Fixed

Anthony Urso

25/Jul/08 23:27

25/Oct/08 21:10

28/Jul/08 14:21

3.0.0

java client

0

1

The ZooKeeper java client main loop crashes on KeeperExceptions. They should be handled when possible.

48053

No Perforce job exists for this issue.

2

33235

28/Jul/08 12:58 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061vj:

ZooKeeper

ZOOKEEPER-104

KeptSet: a distributed data stucture backed by the children of a ZooKeeper node

New Feature

Resolved

Minor

Not A Problem

Anthony Urso

25/Jul/08 19:53

08/Dec/10 15:39

java client

0

1

Here is an implementation of a ZooKeeper backed Java Set. It should be generally useful.

214052

No Perforce job exists for this issue.

4

33590

25/Jul/08 9:07 PM

9 years, 16 weeks, 1 day ago

0|i0642f:

ZooKeeper

ZOOKEEPER-103

Reorganize the ZooKeeper source distro to follow maven conventions

Improvement

Resolved

Major

Won't Do

Hiram R. Chirino

25/Jul/08 17:18

04/Jul/18 11:09

build

0

6

ZOOKEEPER-1078, ZOOKEEPER-80

This was sugested as way to bridge the gap in ZOOKEEPER-83

214051

No Perforce job exists for this issue.

2

68521

25/Jul/08 6:01 PM

1 year, 37 weeks, 1 day ago

0|i0c3k7:

ZooKeeper

ZOOKEEPER-102

Need to replace Jute with supported code

Improvement

Reopened

Major

Unresolved

Jordan Zimmerman

Benjamin Reed

25/Jul/08 16:25

16/Jan/20 20:18

4.0.0

3

26

ZooKeeper currently uses Jute to serialize objects to put on the wire and on disk. We pulled Jute out of Hadoop and added a C binding. Both versions of Jute have evolved (although Hadoop still doesn't have a C binding). It would be nice to use a more standard serialization library. Some options include Thrift or Google's protocol buffers.

Our main requirements would be Java and C bindings and good performance. (For example, serializing to XML would give us incredibly bad performance and would not be acceptible!)

214050

No Perforce job exists for this issue.

0

68522

26/Jul/08 8:21 PM

8 weeks, 6 days ago

0|i0c3kf:

ZooKeeper

ZOOKEEPER-101

Integrate ZooKeeper with "violations" feature on hudson

Improvement

Closed

Major

Fixed

Patrick D. Hunt

25/Jul/08 14:27

25/Oct/08 21:10

08/Aug/08 17:01

3.0.0

build

0

1

ZOOKEEPER-74

"violations" integrates into hudson and supports many more tools than just checkstyle.

http://hudson.gotdns.com/wiki/display/HUDSON/Violations
http://hudson.zones.apache.org/hudson/view/Hadoop/job/Hadoop-trunk/

48054

No Perforce job exists for this issue.

0

33591

11 years, 22 weeks, 4 days ago

0|i0642n:

ZooKeeper

ZOOKEEPER-100

Avoid extending Thread in the public ZooKeeper API

Improvement

Open

Major

Unresolved

Unassigned

Hiram R. Chirino

25/Jul/08 13:14

0

2

This was discussed in ZOOKEEPER-82
This would allow proper checked exceptions to get thrown when the objects are started/stopped.

214049

No Perforce job exists for this issue.

0

68523

11 years, 35 weeks, 6 days ago

0|i0c3kn:

ZooKeeper

ZOOKEEPER-99

All MXBeans interfaces that don't use complex paramters need to be renamed as MBean interaces.

Bug

Open

Major

Unresolved

Unassigned

Hiram R. Chirino

25/Jul/08 12:55

23/Nov/09 07:10

jmx

0

2

All the MXBean interfaces that I've looked at are standard MBean interfaces. The interface names should get renamed to MBean instaead of MXBean. That way the server can also run on a the Java 1.5 Platform.

214048

No Perforce job exists for this issue.

1

33236

27/Feb/09 6:43 AM

10 years, 18 weeks, 3 days ago

0|i061vr:

ZooKeeper

ZOOKEEPER-98

Make the ZooKeeper optional bits seperate jars.

Improvement

Resolved

Major

Done

Unassigned

Hiram R. Chirino

25/Jul/08 10:30

22/Feb/19 08:07

build

0

2

ZOOKEEPER-3021

Optional bits like the ZooKeeper jmx module and the upcoming protocol and spring support stuff should get packaged as separate jars so that they don't keep bloating up the main ZooKeeper jar.

214047

No Perforce job exists for this issue.

0

68524

25/Jul/08 12:55 PM

1 year, 3 weeks, 6 days ago

0|i0c3kv:

ZooKeeper

ZOOKEEPER-97

The code generators should support an optional output directory

Improvement

Closed

Major

Fixed

Hiram R. Chirino

25/Jul/08 10:27

25/Oct/08 21:10

25/Jul/08 17:21

3.0.0

0

1

Currently the code generators (jute compiler and version gen) generate code the current directory. If these tools are to be re-used by smart ides, ant tasks, or maven plugins, the output directory needs to be parameter to the tools.

48055

No Perforce job exists for this issue.

1

33592

25/Jul/08 12:53 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0642v:

ZooKeeper

ZOOKEEPER-96

The jute parser should get generated from the jj files instead of checking in the generated sources

Improvement

Resolved

Major

Fixed

Thomas Koch

Hiram R. Chirino

25/Jul/08 10:24

29/Jun/16 11:15

14/Sep/11 18:38

3.5.0

build

0

2

ZOOKEEPER-1078

48056

No Perforce job exists for this issue.

2

33593

25/Jul/08 12:53 PM

8 years, 28 weeks ago

Reviewed

0|i06433:

ZooKeeper

ZOOKEEPER-95

There is no need to include the code generators in the zookeeper jar

Improvement

Resolved

Major

Done

Unassigned

Hiram R. Chirino

25/Jul/08 10:23

22/Feb/19 08:05

build

0

2

ZOOKEEPER-537, ZOOKEEPER-3021

214046

No Perforce job exists for this issue.

0

68525

25/Jul/08 12:51 PM

1 year, 3 weeks, 6 days ago

0|i0c3l3:

ZooKeeper

ZOOKEEPER-94

JMX tests are needed to verify that the JMX MBeans work properly

Bug

Closed

Major

Fixed

Patrick D. Hunt

Hiram R. Chirino

25/Jul/08 10:22

08/Jul/09 16:23

14/May/09 16:15

3.1.0, 3.1.1

3.2.0

jmx

0

1

ZOOKEEPER-82

48057

No Perforce job exists for this issue.

1

33237

25/Jul/08 12:50 PM

10 years, 46 weeks ago

Reviewed

0|i061vz:

ZooKeeper

ZOOKEEPER-93

Create Documentation for Zookeeper

Task

Closed

Major

Fixed

Patrick D. Hunt

Robbie Scott

24/Jul/08 13:49

25/Oct/08 21:10

24/Sep/08 21:06

3.0.0

documentation

0

60

0%

N/A

Initial version of the documentation, migrated from source forge. Reorganizing, expanding. Initial goal is to port Source Forge documentation into Forrest.

0%

60

48058

No Perforce job exists for this issue.

1

33594

24/Sep/08 8:33 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0643b:

ZooKeeper

ZOOKEEPER-92

ZOOKEEPER-78 spring factory beans for ZooKeeper, ZooKeeperFacade and ZooKeeperServer

Sub-task

Open

Major

Unresolved

Unassigned

James Strachan

24/Jul/08 07:21

10/Jul/11 07:42

java client

1

3

for folks who use Spring for Dependency Injection it might be handy to have a couple of factory beans to make it easier to create and configure the ZooKeeper, ZooKeeperFacade and ZooKeeperServer via the normal Spring dependency mechanism; via Java or XML code etc

68623

No Perforce job exists for this issue.

3

68526

09/Jun/11 5:11 PM

8 years, 37 weeks, 4 days ago

0|i0c3lb:

ZooKeeper

ZOOKEEPER-91

ZOOKEEPER-78 provide an option for the WriteLock to also watch the locks own znode, so that if someone else deletes it then it is equivalent to calling WriteLock.unlock()

Sub-task

Open

Major

Unresolved

Unassigned

James Strachan

24/Jul/08 07:08

05/Aug/08 09:49

java client

0

Most clients probably wont need this, but it could be a handy system management feature to allow the WriteLock to watch its own znode so that if someone else deletes it, it then relinquishes the lock and tries to get it back again

214045

No Perforce job exists for this issue.

0

68527

05/Aug/08 9:49 AM

11 years, 34 weeks, 2 days ago

0|i0c3lj:

ZooKeeper

ZOOKEEPER-90

ZOOKEEPER-78 invoke WhenOwnerListener.whenNotOwner() when the ZK session expires and the znode is the leader

Sub-task

Closed

Major

Fixed

James Strachan

24/Jul/08 05:46

25/Oct/08 21:10

24/Jul/08 06:31

3.0.0

0

48059

No Perforce job exists for this issue.

0

33595

25/Oct/08 9:10 PM

11 years, 22 weeks, 4 days ago

0|i0643j:

ZooKeeper

ZOOKEEPER-89

ZOOKEEPER-78 invoke WhenOwnerListener.whenNotOwner() when the ZK connection fails

Sub-task

Closed

Major

Fixed

James Strachan

24/Jul/08 05:44

25/Oct/08 21:10

24/Jul/08 07:14

3.0.0

0

48060

No Perforce job exists for this issue.

0

33596

25/Oct/08 9:10 PM

11 years, 22 weeks, 4 days ago

0|i0643r:

ZooKeeper

ZOOKEEPER-88

ZOOKEEPER-78 implement java.util.concurrent.locks.Lock

Sub-task

Open

Major

Unresolved

James Strachan

24/Jul/08 05:29

24/Jul/08 16:57

java client

0

we should implement the [java.util.concurrent.locks.Lock|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/locks/Lock.html] to make it easier for folks to reuse the Lock and to help make the API be more natural to end users

214044

No Perforce job exists for this issue.

0

68528

24/Jul/08 4:57 PM

11 years, 36 weeks ago

0|i0c3lr:

ZooKeeper

ZOOKEEPER-87

Follower does not shut itself down if its too far behind the leader.

Bug

Closed

Critical

Fixed

Germán Blanco

Mahadev Konar

23/Jul/08 14:29

13/Mar/14 14:16

26/Sep/13 08:37

3.4.5, 3.5.0

3.4.6, 3.5.0

quorum

1

7

Currently, the follower if lagging behind keeps sending pings to the leader it will stay alive and will keep getting further and further behind the leader. The follower should shut itself down if it is not able to keep up to the leader within some limit so that gurantee of updates can be made to the clients connected to different servers.

patch

71228

No Perforce job exists for this issue.

4

33238

24/Jul/08 6:26 AM

6 years, 2 weeks ago

The value of syncLimit should be reviewed before this change is introduced in the cluster. Since it now detects followers that are lagging behind, clusters in which this was happening will now see followers being disconnected from time to time. This will normally be solved by increasing the syncLimit value.

0|i061w7:

ZooKeeper

ZOOKEEPER-86

intermittent test failure of org.apache.zookeeper.test.AsyncTest

Bug

Closed

Major

Fixed

James Strachan

23/Jul/08 12:59

26/Mar/10 13:24

15/Jan/10 17:02

3.3.0

tests

0

2

ZOOKEEPER-63, ZOOKEEPER-111

OS X and linux. It sometimes passes; but mostly seems to fail on OS X each time

Will attach the test output in an attachment...

48061

No Perforce job exists for this issue.

2

33239

24/Jul/08 6:01 PM

10 years, 10 weeks, 6 days ago

0|i061wf:

ZooKeeper

ZOOKEEPER-85

register the ZooKeeper mailing lists with nabble.com

Task

Closed

Major

Fixed

Patrick D. Hunt

James Strachan

23/Jul/08 12:33

25/Oct/08 21:10

23/Jul/08 14:22

3.0.0

0

Many apache projects including Hadoop register with nabble to host
online forums & great online archives of the mailing lists...
http://www.nabble.com/Hadoop-f17066.html
Currently there's hadoop-core, hbase and lucene on there.

I often refer to mailing list posts by nabble link; they're really
handy. Plus end users often prefer the forum style nabble approach to
getting every single email sent to a mailing list.

Does a committer on zookeeper fancy registering the ZK mailing lists
too (as a child of the Hadoop list)? I'd do it myself but then I'd be
the owner of the forum which doesn't feel right - a committer should
probably do it.

Its a pretty quick process, click here
http://n2.nabble.com/more/MailingListRequest.jtp

and fill out the details. The hardest part is knowing the mailing list
software which AFAIK is ezmlm :)

48062

No Perforce job exists for this issue.

0

33597

23/Jul/08 2:22 PM

11 years, 22 weeks, 4 days ago

0|i0643z:

ZooKeeper

ZOOKEEPER-84

provide a mechanism to reconnect a ZooKeeper if a client receives a SessionExpiredException

Improvement

Resolved

Major

Won't Fix

James Strachan

22/Jul/08 09:27

04/Aug/08 13:37

03/Aug/08 23:23

java client

0

1

am about to attach a patch which adds a reconnect() method to easily re-establish a connection if a session expires - along with a toString() implementation for easier debugging

214043

No Perforce job exists for this issue.

1

33598

22/Jul/08 1:15 PM

11 years, 34 weeks, 3 days ago

0|i06447:

ZooKeeper

ZOOKEEPER-83

Switch to using maven to build ZooKeeper

Improvement

Resolved

Major

Duplicate

Hiram R. Chirino

21/Jul/08 22:29

04/Jul/18 11:05

06/Apr/14 21:34

build

4

12

ZOOKEEPER-2158

Maven is a great too for building java projects at the ASF. It helps standardize the build a bit since it's a convention oriented.
It's dependency auto downloading would remove the need to store the dependencies in svn, and it will handle many of the suggested ASF policies like gpg signing of the releases and such.

The ZooKeeper build is almost vanilla except for the jute compiler bits. Things that would need to change are:
* re-organize the source tree a little so that it uses the maven directory conventions
* seperate the jute bits out into seperate modules so that a maven plugin can be with it

214042

No Perforce job exists for this issue.

0

68529

22/Jul/08 12:24 AM

5 years, 50 weeks, 3 days ago

0|i0c3lz:

ZooKeeper

ZOOKEEPER-82

Make the ZooKeeperServer more DI friendly

Improvement

Closed

Major

Fixed

Hiram R. Chirino

21/Jul/08 15:14

01/Aug/17 09:42

15/Aug/08 13:25

3.0.0

server

0

2

ZOOKEEPER-2861, ZOOKEEPER-94

Proposed changes were discussed in [this mailing list thread|http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/200807.mbox/%3Caf2843cd0807180907v44b310bg232be99ac0b47a27@mail.gmail.com%3E]:

Basic goals are:
* Decouple the current configuration system from the public API. I
see stuff like ZooKeeperServer being coupled to ServerConfig a bit.
* Allow the use of setter injection in addition to constructor
injection. This is the most important thing needed to let spring more
easily configure the objects.
* Move the main() methods out of the ZooKeeperServer class.

48063

No Perforce job exists for this issue.

2

33599

21/Jul/08 7:19 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0644f:

ZooKeeper

ZOOKEEPER-81

JMX module is using 1 java 6 method that has a java 5 equivalent

Improvement

Closed

Major

Fixed

Hiram R. Chirino

18/Jul/08 12:16

23/Jul/08 17:48

22/Jul/08 15:39

3.0.0

server

0

1

It would be nice if the jmx module compiled and ran on java 5 too.

48064

No Perforce job exists for this issue.

1

33600

18/Jul/08 12:37 PM

11 years, 36 weeks, 1 day ago

Reviewed

0|i0644n:

ZooKeeper

ZOOKEEPER-80

Document process for client recipe contributions

Task

Open

Major

Unresolved

Patrick D. Hunt

17/Jul/08 13:37

16/Jan/09 14:40

documentation

0

ZOOKEEPER-79, ZOOKEEPER-103

Per Doug's suggestion I'll use a link instead of copy/paste:

http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-dev/200807.mbox/%3C487F8262.9020001@yahoo-inc.com%3E

214041

No Perforce job exists for this issue.

0

68530

17/Jul/08 1:49 PM

11 years, 10 weeks, 6 days ago

0|i0c3m7:

ZooKeeper

ZOOKEEPER-79

Document jacob's leader election on the wiki recipes page

New Feature

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

17/Jul/08 12:43

25/Oct/08 21:10

26/Sep/08 13:41

3.0.0

documentation

0

ZOOKEEPER-78, ZOOKEEPER-80

The following discussion occurred on the zookeeper-user list. We need to formalize this recipe and document on the wiki recipes page:

---------------------from jacob ----------------
Avinash

The following protocol will help you fix the observed misbehavior. As Flavio points out, you cannot rely on the order of nodes in getChildren, you must use an intrinsic property of each node to determine who is the leader. The protocol devised by Runping Qi and described here will do that.

First of all, when you create child nodes of the node that holds the leadership bids, you must create them with the EPHEMERAL and SEQUENCE flag. ZooKeeper guarantees to give you an ephemeral node named uniquely and with a sequence number larger by at least one than any previously created node in the sequence. You provide a prefix, like "L_" or your own choice, and ZooKeeper creates nodes named "L_23", "L_24", etc. The sequence number starts at 0 and increases monotonously.

Once you've placed your leadership bid, you search backwards from the sequence number of *your* node to see if there are any preceding (in terms of the sequence number) nodes. When you find one, you place a watch on it and wait for it to disappear. When you get the watch notification, you search again, until you do not find a preceding node, then you know you're the leader. This protocol guarantees that there is at any time only one node that thinks it is the leader. But it does not disseminate information about who is the leader. If you want everyone to know who is the leader, you can have an additional Znode whose value is the name of the current leader (or some identifying information on how to contact the leader, etc.). Note that this cannot be done atomically, so by the time other nodes find out who the leader is, the leadership may already have passed on to a different node.

Flavio

Might it make sense to provide a standardized implementation of leader election in the library code in Java?

--Jacob

From: zookeeper-user-bounces@lists.sourceforge.net [mailto:zookeeper-user-bounces@lists.sourceforge.net] On Behalf Of Flavio Junqueira
Sent: Friday, July 11, 2008 1:02 AM
To: zookeeper-user@lists.sourceforge.net
Cc: zookeeper-user@hadoop.apache.org
Subject: Re: [Zookeeper-user] Leader election

Hi Avinash, getChildren returns a list in lexicographic order, so if you are updating the children of the election node concurrently, then you may get a different first node with different clients. If you are using the sequence flag to create nodes, then you may consider stripping the prefix of the node name and using the sufix value to determine order.

Hope it helps.

-Flavio

----- Original Message ----
From: Avinash Lakshman <avinash.lakshman@gmail.com>
To: zookeeper-user@lists.sourceforge.net
Sent: Friday, July 11, 2008 7:20:06 AM
Subject: [Zookeeper-user] Leader election

Hi

I am trying to elect leader among 50 nodes. There is always one odd guy who seems to think that someone else distinct from what some other nodes see as leader. Could someone please tell me what is wrong with the following code for leader election:

public void electLeader()
{
ZooKeeper zk = StorageService.instance().getZooKeeperHandle();
String path = "/Leader";
try
{
String createPath = path + "/L-";
LeaderElector.createLock_.lock();
while( true )
{
/* Get all znodes under the Leader znode */
List<String> values = zk.getChildren(path, false);
/*
* Get the first znode and if it is the
* pathCreated created above then the data
* in that znode is the leader's identity.
*/
if ( leader_ == null )
{
leader_ = new AtomicReference<EndPoint>( EndPoint.fromBytes( zk.getData(path + "/" + values.get(0), false, null) ) );
}
else
{
leader_.set( EndPoint.fromBytes( zk.getData(path + "/" + values .get(0), false, null) ) );
/* Disseminate the state as to who the leader is. */
onLeaderElection();
}
logger_.debug("Elected leader is " + leader_ + " @ znode " + ( path + "/" + values.get(0) ) );
Collections.sort(values);
/* We need only the last portion of this znode */
String[] peices = pathCreated_.split("/");
int index = Collections.binarySearch(values, peices[peices.length - 1]);
if ( index > 0 )
{
String pathToCheck = path + "/" + values.get(index - 1);
Stat stat = zk.exists(pathToCheck, true);
if ( stat != null )
{
logger_.debug("Awaiting my turn ...");
condition_.await();
logger_.debug("Checking to see if leader is around ...");
}
}
else
{
break;
}
}
}
catch ( InterruptedException ex )
{
logger_.warn(LogUtil.throwableToString(ex));
}
catch ( KeeperException ex )
{
logger_.warn(LogUtil.throwableToString(ex));
}
finally
{
LeaderElector.createLock_.unlock();
}
}
}

Thanks
Avinash

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08

_______________________________________________
Zookeeper-user mailing list
Zookeeper-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/zookeeper-user

48065

No Perforce job exists for this issue.

0

33601

17/Jul/08 12:47 PM

11 years, 22 weeks, 4 days ago

0|i0644v:

ZooKeeper

ZOOKEEPER-78

added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation

New Feature

Closed

Major

Fixed

Mahadev Konar

James Strachan

17/Jul/08 11:35

08/Jul/09 16:23

23/Apr/09 17:41

3.0.0

3.2.0

java client

1

2

ZOOKEEPER-88, ZOOKEEPER-89, ZOOKEEPER-90, ZOOKEEPER-91, ZOOKEEPER-92

ZOOKEEPER-79

Here's a patch which adds a little WriteLock helper class for performing leader elections or creating exclusive locks in some directory znode. Note its an early cut; am sure we can improve it over time. The aim is to avoid folks having to use the low level ZK stuff but provide a simpler high level abstraction.

48066

No Perforce job exists for this issue.

12

33602

17/Jul/08 1:18 PM

10 years, 48 weeks, 6 days ago

Reviewed

0|i06453:

ZooKeeper

ZOOKEEPER-77

NPE in jute toCSVString code

Bug

Resolved

Major

Duplicate

Unassigned

Patrick D. Hunt

16/Jul/08 12:22

27/Aug/08 14:30

jute

1

2

ZOOKEEPER-16

As reported by user:

From: Martin Schaaf <ms@101tec.com>
List-Id: <zookeeper-user.lists.sourceforge.net>

FYI today I found this NPE in the logs. After this Exception an Error
event was thrown.

java.lang.NullPointerException
at com.yahoo.jute.Utils.toCSVString(Utils.java:128)
at com.yahoo.jute.CsvOutputArchive.writeString(CsvOutputArchive.java:
95)
at com.yahoo.zookeeper.proto.WatcherEvent.toString(WatcherEvent.java:
60)
at net.sf.katta.zk.ZKClient.process(ZKClient.java:404)
at com.yahoo.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:264)

214040

No Perforce job exists for this issue.

0

33240

23/Jul/08 2:18 AM

11 years, 31 weeks, 1 day ago

0|i061wn:

ZooKeeper

ZOOKEEPER-76

Commit 677109 removed the cobertura library, but not the build targets.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Jakob Homan

15/Jul/08 22:30

25/Oct/08 21:10

16/Jul/08 14:39

3.0.0

build

0

1

Cobertura library removed due to licensing issues, but the code-coverage ant targets were left in. Have deleted them and created patch.

48067

No Perforce job exists for this issue.

3

33241

16/Jul/08 12:18 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061wv:

ZooKeeper

ZOOKEEPER-75

cleanup the library directory

Bug

Closed

Major

Fixed

Patrick D. Hunt

Owen O'Malley

15/Jul/08 18:50

25/Oct/08 21:10

15/Jul/08 19:51

3.0.0

build

0

1

The cobertura library is GPL and must be removed from the code base. All of the X.jar files should have matching X.license files that contain their license. There is no license for junit.

48068

No Perforce job exists for this issue.

3

33242

15/Jul/08 6:57 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061x3:

ZooKeeper

ZOOKEEPER-74

Cleaning/restructuring up Zookeeper server code

Improvement

Resolved

Major

Won't Fix

Mahadev Konar

14/Jul/08 19:29

23/Apr/14 16:16

3.5.0

server

0

2

ZOOKEEPER-101

I have been thinking this for a while and find that the zookeeper server code needs some cleaning up. The server code is a little tricky/confusing to read sometimes gievn that there is no clearity on ownership of objects. I will put down a proposal for restructuring/cleaning the code up with javadocs so that the code is easier to understand and develop on. comments on what you find confusing are welcome on this jira.

70750

No Perforce job exists for this issue.

0

68531

24/Jul/08 12:14 PM

5 years, 48 weeks, 1 day ago

not a blocker for 3.2, moving to 3.3

0|i0c3mf:

ZooKeeper

ZOOKEEPER-73

Move ZK wiki from SourceForge to Apache

Task

Closed

Major

Fixed

Patrick D. Hunt

08/Jul/08 13:30

25/Oct/08 21:10

10/Oct/08 14:14

3.0.0

documentation

0

ZOOKEEPER-181

Move the ZK wiki content from sourceforge to apache wiki.

base Apache page is:

http://wiki.apache.org/hadoop/ZooKeeper

48069

No Perforce job exists for this issue.

0

33603

11 years, 22 weeks, 4 days ago

Reviewed

0|i0645b:

ZooKeeper

ZOOKEEPER-72

Initial creation/setup of ZooKeeper ASF site.

Task

Closed

Major

Fixed

Patrick D. Hunt

08/Jul/08 13:10

25/Oct/08 21:10

22/Jul/08 18:01

3.0.0

documentation

0

ZOOKEEPER-70

Using the output of ZOOKEEPER-70 generate the ZooKeeper apache.org site.

Out structure/process is based on Hadoop's "how to release" document:
http://wiki.apache.org/hadoop/HowToRelease

48070

No Perforce job exists for this issue.

1

33604

11 years, 22 weeks, 4 days ago

0|i0645j:

ZooKeeper

ZOOKEEPER-71

Determine what to do re ZooKeeper Changelog(s)

Task

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

08/Jul/08 13:05

25/Oct/08 21:10

27/Aug/08 15:05

3.0.0

0

This JIRA is to collect feedback and document what we will be doing re changelog file(s) in zookeeper SVN repository/release.

See ZOOKEEPER-45 for some background, summary:

* suggestion was made to have single CHANGES.txt, similar to hadoop core/hbase
However a concern was brought up that it would be better to have 2 files - one for java code (client & server) and one for C client. This would make it easier for users to discern (C client users would see it broken out in separate file)

Please add comments/votes for what you think we should do.

48071

No Perforce job exists for this issue.

0

33605

27/Aug/08 3:05 PM

11 years, 22 weeks, 4 days ago

0|i0645r:

ZooKeeper

ZOOKEEPER-70

Add skeleton forrest doc structure for ZooKeeper

Task

Closed

Major

Fixed

Patrick D. Hunt

08/Jul/08 13:01

25/Oct/08 21:10

22/Jul/08 18:00

3.0.0

documentation

0

ZOOKEEPER-72

Add a skeleton forrest doc structure for ZooKeeper.

Doug has suggested using hbase as a starting point.

Note: I noticed that forrest doesn't work with java6, it will run with java5.

48072

No Perforce job exists for this issue.

3

33606

08/Jul/08 9:09 PM

11 years, 22 weeks, 4 days ago

0|i0645z:

ZooKeeper

ZOOKEEPER-69

ZooKeeper logo

Wish

Closed

Minor

Fixed

Benjamin Reed

Flavio Paiva Junqueira

08/Jul/08 05:15

13/Feb/09 16:18

02/Feb/09 21:37

3.1.0

documentation

0

I think we need a cool logo for the project. The ones I've seen so far are a little lame, and that includes the one I've created for SourceForge. If anyone on this list has an idea or knows of anyone with some art skills, plese add a commento to this Jira.

48073

No Perforce job exists for this issue.

4

33607

08/Jul/08 1:36 PM

11 years, 8 weeks, 2 days ago

Reviewed

0|i06467:

ZooKeeper

ZOOKEEPER-68

parseACLs in ZooKeeper.java fails to parse elements of ACL, should be lastIndexOf rather than IndexOf

Bug

Closed

Minor

Fixed

Mahadev Konar

Jakob Homan

07/Jul/08 18:11

25/Oct/08 21:10

07/Jul/08 19:03

3.0.0

java client

0

1

the parseACLs(String aclString) method attempts to prase ACLs from the form of scheme:id:perm into the three components, delineated by the colons. The current version calls indexOf for both the first and second colon, receiving the same value for both and failing as if there were only one colon in the string. I created a one-line patch to call lastIndexOf for the second colon.

48074

No Perforce job exists for this issue.

2

33243

07/Jul/08 6:39 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061xb:

ZooKeeper

ZOOKEEPER-67

Add a two argument constructor to ZooKeeper that uses a NullWatcher for the default Watcher

Improvement

Resolved

Minor

Won't Fix

Unassigned

Anthony Urso

03/Jul/08 21:59

22/Jul/08 13:46

java client

0

ZOOKEEPER-66

This patch adds a two argument constructor to ZooKeeper that uses a NullWatcher for the default Watcher.

214039

No Perforce job exists for this issue.

1

33608

21/Jul/08 7:24 PM

11 years, 36 weeks, 2 days ago

0|i0646f:

ZooKeeper

ZOOKEEPER-66

Add NullWatcher, a watcher that does nothing

New Feature

Resolved

Minor

Won't Fix

Unassigned

Anthony Urso

03/Jul/08 21:51

22/Jul/08 13:46

java client

0

ZOOKEEPER-67

I found that I was repeating this code all over the place. Here is a NullWatcher class, a Watcher that does nothing at all.

214038

No Perforce job exists for this issue.

1

33609

06/Jul/08 8:42 AM

11 years, 36 weeks, 2 days ago

0|i0646n:

ZooKeeper

ZOOKEEPER-65

Log information on connection properties

Improvement

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

02/Jul/08 12:34

14/Dec/19 06:08

3.7.0

c client, java client, server

0

2

We should log information on the connection to enable better field issue debugging.

In particular if we logged the connection latency (time for req->resp) during session establishment this would give some information about the connection. We should log the initial reading at INFO level, then log subsequent information at trace level.

Would be nice to log trace events for client/server pings for example.

70772

No Perforce job exists for this issue.

0

68532

11 years, 39 weeks, 1 day ago

not a blocker for 3.2, moving to 3.3

0|i0c3mn:

ZooKeeper

ZOOKEEPER-64

We should log system env information when intializing client/server

Improvement

Closed

Major

Fixed

Patrick D. Hunt

02/Jul/08 12:30

13/Feb/09 16:18

01/Dec/08 19:14

3.1.0

c client, java client, server

0

We should log system environment information (as much as possible) during both client & server initialization. This will help with debugging of customer issues.

48075

No Perforce job exists for this issue.

1

33610

18/Nov/08 9:50 PM

11 years, 17 weeks, 1 day ago

Reviewed

0|i0646v:

ZooKeeper

ZOOKEEPER-63

Race condition in client close() operation

Bug

Closed

Major

Fixed

Patrick D. Hunt

30/Jun/08 18:08

25/Oct/08 21:10

27/Aug/08 19:39

3.0.0

java client

0

3

ZOOKEEPER-126, ZOOKEEPER-86

There is a race condition in the java close operation on ZooKeeper.java.

Client is sending a disconnect request to the server. Server will close any open connections with the client when it receives this. If the client has not yet shutdown it's subthreads (event/send threads for example) these threads may consider the condition an error. We see this alot in the tests where the clients output error logs because they are unaware that a disconnection has been requested by the client.

Ben mentioned: perhaps we just have to change state to closed (on client) before sending disconnect request.

48076

No Perforce job exists for this issue.

2

33244

23/Jul/08 12:04 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061xj:

ZooKeeper

ZOOKEEPER-62

Generally improve logging to enable debuggability in the field.

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

30/Jun/08 18:03

07/May/10 13:40

15/Jan/10 17:00

3.3.0

c client, java client, server

1

2

We need to improve our logging to enable debugging of field issues.

Flavio, assigning to you as you are currently looking at some client/server issues that could benefit from better logging. Please attach patches if you see potential areas for improvement.

logging

48077

No Perforce job exists for this issue.

1

33611

03/Oct/08 1:58 PM

10 years, 10 weeks, 6 days ago

not a blocker for 3.2, moving to 3.3

0|i06473:

ZooKeeper

ZOOKEEPER-61

Address (remove) use of sleep(#) in client/server test cases.

Improvement

Closed

Major

Fixed

Patrick D. Hunt

30/Jun/08 17:55

25/Oct/08 21:10

25/Aug/08 19:54

3.0.0

tests

0

ZOOKEEPER-54, ZOOKEEPER-111

Many of the java client/server test cases use "sleep(x)" during initial rendezvous. This makes the tests slow, very brittle and they tend to fail occasionally during CI (depending on load). In particular I've seen QuorumTest exhibit this.

Would also be nice if we could extract the "base" client/server test functionality to enable re-use and make test writing easier. I started this with ClientBase in tests but could do a better job.

48078

No Perforce job exists for this issue.

0

33612

03/Jul/08 1:21 PM

11 years, 22 weeks, 4 days ago

0|i0647b:

ZooKeeper

ZOOKEEPER-60

Get cppunit tests running as part of Hudson CI

Improvement

Closed

Major

Fixed

Giridharan Kesavan

Patrick D. Hunt

30/Jun/08 17:51

08/Jul/09 16:23

02/Apr/09 17:31

3.0.0, 3.0.1, 3.1.0, 3.1.1

3.2.0

build

0

1

Investigate if it is possible to run cppunit tests as part of Hudson.

48079

No Perforce job exists for this issue.

4

33613

25/Mar/09 12:14 AM

10 years, 51 weeks, 5 days ago

Get cppunit tests running from ant.

Reviewed

0|i0647j:

ZooKeeper

ZOOKEEPER-59

Synchronized block in NIOServerCnxn

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

30/Jun/08 07:20

26/Mar/10 13:24

08/Mar/10 21:01

3.3.0

server

0

4

There are two synchronized blocks locking on different objects, and to me they should be guarded by the same object. Here are the parts of the code I'm talking about:

{noformat}
NIOServerCnxn.readRequest@444
...
synchronized (this) {
outstandingRequests++;
// check throttling
if (zk.getInProcess() > factory.outstandingLimit) {
disableRecv();
// following lines should not be needed since we are already
// reading
// } else {
// enableRecv();
}
}
{noformat}

{noformat}
NIOServerCnxn.sendResponse@740
...
synchronized (this.factory) {
outstandingRequests--;
// check throttling
if (zk.getInProcess() < factory.outstandingLimit
|| outstandingRequests < 1) {
sk.selector().wakeup();
enableRecv();
}
}
{noformat}

I think the second one is correct, and the first synchronized block should be guarded by "this.factory".

This could be related to issue ZOOKEEPER-57, but I have no concrete indication that this is the case so far.

48080

No Perforce job exists for this issue.

4

33245

30/Jun/08 11:06 AM

10 years, 2 weeks, 3 days ago

not a blocker for 3.2, moving to 3.3

Reviewed

0|i061xr:

ZooKeeper

ZOOKEEPER-58

Race condition on ClientCnxn.java

Bug

Closed

Major

Fixed

Benjamin Reed

Flavio Paiva Junqueira

27/Jun/08 11:38

25/Oct/08 21:10

27/Jul/08 04:07

3.0.0

java client

0

1

There is a race condition involving the ByteByffer incomingBuffer, a field of ClientCnxn.SendThread. SendThread reads a packet in two steps: first it reads the length of the packet, followed by a read of the packet itself. Each of these steps corresponds to a call to doIO() from the main loop of run(). If there is an exception or the session times out, then it may leave incomingBuffer in an inconsistent state.

The attached patch adds code to reset incomingBuffer upon a call to SendThread.cleanup(). This method is called upon an exception on run().

48081

No Perforce job exists for this issue.

1

33246

27/Jun/08 12:08 PM

11 years, 22 weeks, 4 days ago

0|i061xz:

ZooKeeper

ZOOKEEPER-57

Connection times out when idle

Bug

Resolved

Major

Invalid

Flavio Paiva Junqueira

27/Jun/08 04:16

16/Jan/09 03:44

16/Jan/09 01:19

java client

3

I have observed several connection timeouts with one single client, during periods of inactivity. From reports of other users, it seems that others are observing a similar issue.

In my case, tickTime is 2000, and the client sets session timeout to be 10000. Client and servers are on different sites. I suspect there is something wrong with the mechanism that issues ping messages.

214037

No Perforce job exists for this issue.

0

33247

27/Jun/08 3:06 PM

11 years, 10 weeks, 6 days ago

0|i061y7:

ZooKeeper

ZOOKEEPER-56

Add clover support to build.xml

Improvement

Closed

Major

Fixed

Patrick D. Hunt

26/Jun/08 18:26

25/Oct/08 21:10

13/Aug/08 17:11

3.0.0

build

0

Clover is what ASF uses for code coverage. We currently have cobertura, but need to add support for clover (keep the existing cobertura support for individual developers to use). Clover is run automatically by ASF Hudson.

Hadoop has support for clover in their build.xml, port their clover related targets into zookeeper build.xml

48082

No Perforce job exists for this issue.

2

33614

13/Aug/08 3:51 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0647r:

ZooKeeper

ZOOKEEPER-55

build.xml failes to retrieve a release number from SVN and the ant target "dist" fails

Bug

Closed

Major

Fixed

Andrew Kornev

26/Jun/08 17:13

25/Oct/08 21:10

26/Jun/08 19:50

3.0.0

build

0

1

Due to the recent directory layout change,
1) the ant target "svn-revision" fails to retrieve the release number,
2) the ant target "dist" fails to create the distribution package.

48083

No Perforce job exists for this issue.

1

33248

26/Jun/08 5:28 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061yf:

ZooKeeper

ZOOKEEPER-54

remove sleeps in the tests.

Bug

Closed

Major

Fixed

Patrick D. Hunt

Mahadev Konar

25/Jun/08 19:30

25/Oct/08 21:10

25/Aug/08 19:53

3.0.0

tests

0

1

ZOOKEEPER-61, ZOOKEEPER-111

remove the sleeps in the tests.

48084

No Perforce job exists for this issue.

0

33249

08/Aug/08 4:57 PM

11 years, 22 weeks, 4 days ago

0|i061yn:

ZooKeeper

ZOOKEEPER-53

tests failing on solaris.

Bug

Closed

Blocker

Fixed

Mahadev Konar

25/Jun/08 18:43

25/Oct/08 21:10

25/Jun/08 19:06

3.0.0

0

1

tests are failing on solaris and randomly on my machine as well.

48085

No Perforce job exists for this issue.

1

33250

25/Jun/08 6:52 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061yv:

ZooKeeper

ZOOKEEPER-52

Session ids on the stat cmd information

Improvement

Open

Minor

Unresolved

Unassigned

Benjamin Reed

25/Jun/08 11:34

server

0

3

Currently there is not a way to correlate session ids with client connections. It would be nice if the stat command listed the session id along with the ip address.

214036

No Perforce job exists for this issue.

0

68533

11 years, 40 weeks, 1 day ago

0|i0c3mv:

ZooKeeper

ZOOKEEPER-51

Review error handling in PrepRequestProcessor.fixupACL

Improvement

Open

Minor

Unresolved

Benjamin Reed

Patrick D. Hunt

23/Jun/08 19:20

14/Dec/19 06:09

3.7.0

server

0

Line 409 (fixupacl method) logs error for "missing authenciation provider...", is this really an error? (no exception thrown as a result...) should we be notifying the client in this case (might help with client side debugging.

68114

No Perforce job exists for this issue.

0

68534

05/Jan/09 12:27 PM

8 years, 37 weeks, 1 day ago

0|i0c3n3:

ZooKeeper

ZOOKEEPER-50

Watch event delivery rules

Bug

Closed

Major

Fixed

Benjamin Reed

20/Jun/08 16:36

01/May/13 22:29

15/Jan/10 16:58

3.0.0

java client

0

4

ZOOKEEPER-23, ZOOKEEPER-39

We need to make clear in the documentation and enforce in the code the following watch event rules:

# A watch event will be delivered once to each watcher, even if it is registered multiple times. For example, if the same watch object is used for getChildren("/foo", watchObj) and getData("/foo", watchObj, stat) and foo is deleted, watchObj will be called once to processed the NodeDeleted event.
# Session events will be delivered to all watchers.

*Note: a watcher is a Watcher object in Java or a (watch function, context) pair in C.*

There is currently a bug in the Java client that causes the session disconnected event to be delivered twice to the default watcher if the default watcher is also used to watch a path. This violates rule 1.

48086

No Perforce job exists for this issue.

0

33251

15/Jul/08 8:05 PM

10 years, 10 weeks, 6 days ago

0|i061z3:

ZooKeeper

ZOOKEEPER-49

SetACL does not work

Bug

Closed

Major

Fixed

Benjamin Reed

20/Jun/08 03:23

01/May/13 22:29

05/Aug/08 09:46

3.0.0

server

0

1

ZOOKEEPER-48

As reported by Shane:

Still exploring the ACL stuff in Zookeeper. Tried using setACL for a
path but get InvalidACL error thrown .... looking at pRequest in
PrepRequestProcessor ... and in particular these lines ...

SetACLRequest setAclRequest = new SetACLRequest();
if (!fixupACL(request.authInfo,
setAclRequest.getAcl())) {
throw new KeeperException(Code.InvalidACL);
}

a new SetACLRequest will return a null when called in fixupACL
returning false and throwing the exception .... as far as I can see.

48087

No Perforce job exists for this issue.

0

33252

15/Jul/08 8:03 PM

11 years, 22 weeks, 4 days ago

0|i061zb:

ZooKeeper

ZOOKEEPER-48

AUTH_ID not handled correctly when no auth ids are present

Bug

Closed

Major

Fixed

Benjamin Reed

20/Jun/08 02:02

01/May/13 22:29

31/Jul/08 16:11

3.0.0

server

0

1

ZOOKEEPER-49

AUTH_ID is used (usually done using Ids.CREATOR_ALL_ACL ) to represent the id that was used to authenticate with ZooKeeper. Thus, an exception should be raised if there are no authenticated ids present. Currently, the exception is not being raised.

48088

No Perforce job exists for this issue.

1

33253

20/Jun/08 1:54 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061zj:

ZooKeeper

ZOOKEEPER-47

io thread still around after a EXPIRED_SESSION or AUTH_FAILED event is received

Bug

Resolved

Major

Invalid

Unassigned

Benjamin Reed

18/Jun/08 17:45

27/Jun/08 10:57

c client

0

If a session expires, the zhandle_t becomes invalid and useless; however, the io thread keeps going.

do_io in mt_adaptor.c needs to check the return code of zookeeper_interest and zookeeper_process and get out of the loop if the handle is invalid.

214035

No Perforce job exists for this issue.

0

33254

26/Jun/08 8:05 PM

11 years, 39 weeks, 6 days ago

0|i061zr:

ZooKeeper

ZOOKEEPER-46

Clients should check the latencies to the various servers and try to connect to the servers with the lowest latencies first

Improvement

Open

Major

Unresolved

Unassigned

Benjamin Reed

17/Jun/08 10:56

17/Jun/08 10:57

c client, java client

0

When ZooKeeper clients and servers are on the same LAN or in the same data center it really doesn't matter which ZooKeeper server they connect to. However, when the ZooKeeper cluster spans data centers, clients should try to connect to local servers first.

214034

No Perforce job exists for this issue.

0

68535

11 years, 41 weeks, 2 days ago

0|i0c3nb:

ZooKeeper

ZOOKEEPER-45

Restructure the SVN repository after initial import

Task

Closed

Major

Fixed

Patrick D. Hunt

13/Jun/08 02:18

25/Oct/08 21:10

08/Jul/08 13:08

3.0.0

build

0

SVN and the ant build have to be updated after the initial import from SourceForge (see INFRA-1644) in order to get us aligned with Apache release process/collateral and some general cleanup of the repository. Please review/comment on the following:

I intend to follow these steps:
1) collapse svn hadoop/zookeeper/trunk/zookeeper/... down to hadoop/zookeeper/trunk/... (remove unnecessary zk dir under trunk)

2) change the java package prefix from com.yahoo to org.apache
3) update the license headers in all source files
4) replace LICENSE file in root dir with Apache LICENSE file
5) add attribution NOTICE file to root dir
6) update the README file appropriately

7) create a new "src" directory in the zk root
8) move root "c" directory into src
9) move root "java" directory into src
10) create root "docs" directory
11) create src/docs directory
12) rename src/java/src to src/java/main
13) move root "test" directory to src/java/test

14) build.xml will be updated appropriately for all the moves. it will also be updated with apache specific targets similar to what exists in hadoop (such as building the forrest documentation).

steps 10&11 are required for zk documentation storage. src/docs is the location of all documentation in "forrest xml" format - the documentation source/originals (what you edit when you change the docs). the ant script is setup to build docs (forrest) into the root docs directory. these files are also committed to svn (both docs and src/docs). later, during a release process, the files in the root docs directory are checked out onto the system hosting the apache web site and pushed to the mirrors.

I'll submit patches for any changes/additions of files. However I suspect that the directory move operations in SVN will not have an accompanying patch - this "script" is the documentation of what will be changed (comments added for any deviation).

48089

No Perforce job exists for this issue.

4

33615

13/Jun/08 6:38 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0647z:

ZooKeeper

ZOOKEEPER-44

DataTree does not use natural sort for getChildren

Improvement

Closed

Minor

Fixed

Jakob Homan

Stu Hood

11/Jun/08 00:17

25/Oct/08 21:10

01/Aug/08 20:09

3.0.0

server

1

2

DataTree.getChildren() performs Collection.sort() on the list of children before returning it, but Java's default comparator for Strings will sort 'lock-20' before 'lock-3' for instance.

48090

No Perforce job exists for this issue.

2

33616

11/Jun/08 10:36 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i06487:

ZooKeeper

ZOOKEEPER-43

Server side of the auto reset watches patch

Improvement

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

10/Jun/08 18:51

25/Oct/08 21:10

21/Oct/08 19:50

3.0.0

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1975262&group_id=209147&atid=1008546

48091

No Perforce job exists for this issue.

6

33617

16/Oct/08 1:50 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0648f:

ZooKeeper

ZOOKEEPER-42

Change LE default to Fast TCP

Improvement

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

10/Jun/08 18:50

25/Oct/08 21:10

31/Jul/08 17:23

3.0.0

leaderElection

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1970356&group_id=209147&atid=1008546

48092

No Perforce job exists for this issue.

1

33618

24/Jul/08 7:23 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0648n:

ZooKeeper

ZOOKEEPER-41

Sample startup script

New Feature

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

10/Jun/08 18:48

25/Oct/08 21:10

03/Jul/08 17:31

3.0.0

scripts

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1951806&group_id=209147&atid=1008546

48093

No Perforce job exists for this issue.

1

33619

30/Jun/08 7:43 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0648v:

ZooKeeper

ZOOKEEPER-40

Go read-only on loss of quorum

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 18:47

24/Dec/13 18:42

quorum

0

3

ZOOKEEPER-270

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1962503&group_id=209147&atid=1008547

214033

No Perforce job exists for this issue.

0

68536

24/Dec/13 5:50 AM

6 years, 13 weeks, 2 days ago

0|i0c3nj:

ZooKeeper

ZOOKEEPER-39

Use Watcher objects rather than boolean on read operations

New Feature

Closed

Major

Fixed

Andrew Kornev

Patrick D. Hunt

10/Jun/08 18:43

25/Jul/08 06:42

24/Jul/08 17:56

3.0.0

c client, java client

0

ZOOKEEPER-50

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1962501&group_id=209147&atid=1008547

48094

No Perforce job exists for this issue.

2

33620

06/Jul/08 10:59 PM

11 years, 35 weeks, 6 days ago

0|i06493:

ZooKeeper

ZOOKEEPER-38

headers (version+) in log/snap files

New Feature

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

10/Jun/08 18:42

25/Oct/08 21:10

01/Oct/08 02:28

3.0.0

server

0

ZOOKEEPER-27, ZOOKEEPER-32

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1961767&group_id=209147&atid=1008547

48095

No Perforce job exists for this issue.

13

33621

11/Jun/08 4:13 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0649b:

ZooKeeper

ZOOKEEPER-37

WebDAV access to ZooKeeper

New Feature

Open

Major

Unresolved

Patrick D. Hunt

10/Jun/08 18:41

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1961764&group_id=209147&atid=1008547

214032

No Perforce job exists for this issue.

0

68537

11 years, 42 weeks, 2 days ago

0|i0c3nr:

ZooKeeper

ZOOKEEPER-36

REST access to ZooKeeper

New Feature

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 18:39

08/Jul/09 16:23

08/May/09 01:28

3.2.0

contrib

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1961763&group_id=209147&atid=1008547

48096

No Perforce job exists for this issue.

9

3171

04/Apr/09 12:19 AM

10 years, 46 weeks, 6 days ago

rest access too zookeeper using jax-rs via jersey.

Reviewed

0|i00wc7:

ZooKeeper

ZOOKEEPER-35

Replay logs

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 18:38

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1958121&group_id=209147&atid=1008547

214031

No Perforce job exists for this issue.

0

68538

11 years, 42 weeks, 2 days ago

0|i0c3nz:

ZooKeeper

ZOOKEEPER-34

Optimize string deserialization

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 18:37

14/Dec/19 06:07

3.7.0

server

0

3

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1952497&group_id=209147&atid=1008547

62182

No Perforce job exists for this issue.

0

68539

06/Sep/11 7:39 AM

8 years, 29 weeks, 2 days ago

not a blocker for 3.2, moving to 3.3

0|i0c3o7:

ZooKeeper

ZOOKEEPER-33

Better ACL management

New Feature

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

10/Jun/08 18:34

07/Sep/11 10:49

14/Oct/08 13:48

3.0.0

server

0

ZOOKEEPER-1173

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1945175&group_id=209147&atid=1008547

48097

No Perforce job exists for this issue.

8

33622

06/Oct/08 6:24 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0649j:

ZooKeeper

ZOOKEEPER-32

CRCs for ZooKeeper data

New Feature

Closed

Major

Fixed

Mahadev Konar

Patrick D. Hunt

10/Jun/08 18:32

25/Oct/08 21:10

01/Oct/08 16:51

3.0.0

server

0

ZOOKEEPER-38

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1945106&group_id=209147&atid=1008547

48098

No Perforce job exists for this issue.

0

33623

24/Jul/08 4:17 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0649r:

ZooKeeper

ZOOKEEPER-31

Need deleteRecursively method

New Feature

Resolved

Major

Duplicate

Unassigned

Patrick D. Hunt

10/Jun/08 18:17

11/Dec/10 22:55

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1941112&group_id=209147&atid=1008547

214030

No Perforce job exists for this issue.

0

33624

11/Dec/10 10:55 PM

9 years, 15 weeks, 4 days ago

0|i0649z:

ZooKeeper

ZOOKEEPER-30

Hooks for atomic broadcast protocol

New Feature

Open

Major

Unresolved

Mahadev Konar

Patrick D. Hunt

10/Jun/08 18:15

11/Apr/13 17:36

quorum

1

6

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1938788&group_id=209147&atid=1008547

214029

No Perforce job exists for this issue.

2

68540

27/Oct/08 10:15 AM

6 years, 50 weeks ago

0|i0c3of:

ZooKeeper

ZOOKEEPER-29

Flexible quorums

New Feature

Closed

Major

Fixed

Flavio Paiva Junqueira

Patrick D. Hunt

10/Jun/08 18:12

08/Jul/09 16:23

24/Apr/09 14:28

3.2.0

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1938782&group_id=209147&atid=1008547

48099

No Perforce job exists for this issue.

16

33625

21/Nov/08 6:53 AM

10 years, 48 weeks, 5 days ago

Flexible quorums in zookeeper.

Reviewed

0|i064a7:

ZooKeeper

ZOOKEEPER-28

Incompatible client and server list detection

New Feature

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 18:09

29/Nov/10 03:57

c client, java client, server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1937084&group_id=209147&atid=1008547

214028

No Perforce job exists for this issue.

0

68541

11/Jun/08 5:05 PM

9 years, 17 weeks, 3 days ago

0|i0c3on:

ZooKeeper

ZOOKEEPER-27

Unique DB identifiers for servers and clients

New Feature

Closed

Major

Fixed

Unassigned

Patrick D. Hunt

10/Jun/08 17:58

06/Feb/17 15:49

01/Oct/08 16:53

3.6.0

c client, java client, server

0

4

ZOOKEEPER-38

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1937075&group_id=209147&atid=1008547

here is the text from sourceforge:

There should be a persistent unique identifier for an instance of ZooKeeper. Currently, if you bring a cluster down without stopping clients and reinitialize the servers, the servers will start logging client zxid errors because the clients have seen a later transaction than the server has. In reality the clients should detect that they are now talking to a new instance of the database and close the session.

A similar problem occurs when a server fails in a cluster of three machines, and the other two machines are reinitialized and restarted. If the failed machine starts up again, there is a chance that the old machine may get elected leader (since it will have the highest zxid) and overwrite new data.

A unique random id should probably get generated when a new cluster comes up. (It is easy to detect since the zxid will be zero.) Leader Election and the Leader should validate that the peers have the same database id. Clients should also validate that they are talking to servers with the same database id during a session.

48100

No Perforce job exists for this issue.

0

33626

01/Oct/08 4:53 PM

3 years, 6 weeks, 3 days ago

Incompatible change

0|i064af:

ZooKeeper

ZOOKEEPER-26

monitoring and management via JMX

New Feature

Closed

Major

Won't Fix

Andrew Kornev

Patrick D. Hunt

10/Jun/08 17:34

11/Aug/08 14:30

24/Jul/08 16:14

3.0.0

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1894138&group_id=209147&atid=1008547

214027

No Perforce job exists for this issue.

0

33627

24/Jul/08 4:14 PM

11 years, 36 weeks ago

0|i064an:

ZooKeeper

ZOOKEEPER-25

FUSE for ZooKeeper

New Feature

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:32

25/Oct/08 21:10

13/Aug/08 13:59

3.0.0

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1873981&group_id=209147&atid=1008547

48101

No Perforce job exists for this issue.

3

33628

02/Jul/08 4:31 PM

11 years, 22 weeks, 4 days ago

0|i064av:

ZooKeeper

ZOOKEEPER-24

Do Application based outstanding request throttling

New Feature

Open

Major

Unresolved

Benjamin Reed

Patrick D. Hunt

10/Jun/08 17:30

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1848004&group_id=209147&atid=1008547

214026

No Perforce job exists for this issue.

0

68542

11 years, 42 weeks, 2 days ago

0|i0c3ov:

ZooKeeper

ZOOKEEPER-23

Auto reset of watches on reconnect

New Feature

Closed

Major

Fixed

Benjamin Reed

Patrick D. Hunt

10/Jun/08 17:29

01/May/13 22:29

21/Oct/08 21:22

3.0.0

c client, java client

0

ZOOKEEPER-50, ZOOKEEPER-138, ZOOKEEPER-149

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1831413&group_id=209147&atid=1008547

18587

No Perforce job exists for this issue.

1

33629

20/Oct/08 7:43 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i064b3:

ZooKeeper

ZOOKEEPER-22

Automatic request retries on connect failover

New Feature

Open

Major

Unresolved

Mahadev Konar

Patrick D. Hunt

10/Jun/08 17:27

14/Dec/19 06:08

3.7.0

c client, java client

2

20

ZOOKEEPER-1905, ZOOKEEPER-835, ZOOKEEPER-660

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1831412&group_id=209147&atid=1008547

When a connection to a ZooKeeper server fails, all of the pending requests
will return an error. In reality the requests should be resubmitted when
the client reestablishes a connection to ZooKeeper.

For read requests, it's no big deal to just reissue the request. For update
requests, the ZooKeeper must be able to detect if the request has been
processed and, if so, return the result of the previous execution;
otherwise, it should process the request.

70731

No Perforce job exists for this issue.

2

68543

24/Jul/08 4:46 AM

4 years, 2 weeks, 2 days ago

0|i0c3p3:

ZooKeeper

ZOOKEEPER-21

Improve zk ctor/watcher (state transition) docs

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:25

25/Oct/08 21:10

26/Sep/08 14:09

3.0.0

java client

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1985723&group_id=209147&atid=1008544

48102

No Perforce job exists for this issue.

1

33255

26/Sep/08 2:00 PM

11 years, 22 weeks, 4 days ago

Reviewed

0|i061zz:

ZooKeeper

ZOOKEEPER-20

Child watches are not triggered when the node is deleted

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:23

25/Oct/08 21:10

27/Aug/08 14:36

3.0.0

c client, java client, server

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1981340&group_id=209147&atid=1008544

48103

No Perforce job exists for this issue.

0

33256

11 years, 22 weeks, 4 days ago

0|i06207:

ZooKeeper

ZOOKEEPER-19

Vector of Integers with Jute

Bug

Closed

Major

Later

Flavio Paiva Junqueira

Patrick D. Hunt

10/Jun/08 17:22

09/Sep/08 17:10

09/Sep/08 17:06

jute

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1981288&group_id=209147&atid=1008544

214025

No Perforce job exists for this issue.

0

33257

26/Aug/08 4:18 AM

11 years, 29 weeks, 2 days ago

0|i0620f:

ZooKeeper

ZOOKEEPER-18

keeper state inconsistency

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:20

25/Oct/08 21:10

25/Sep/08 18:33

3.0.0

java client

1

2

ZOOKEEPER-139, ZOOKEEPER-7

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1979772&group_id=209147&atid=1008544

48104

No Perforce job exists for this issue.

0

33258

11 years, 22 weeks, 4 days ago

Incompatible change

0|i0620n:

ZooKeeper

ZOOKEEPER-17

zookeeper_init doc needs clarification

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:18

25/Oct/08 21:10

10/Oct/08 18:56

3.0.0

c client, documentation

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1967467&group_id=209147&atid=1008544

48105

No Perforce job exists for this issue.

2

33259

01/Oct/08 3:23 AM

11 years, 22 weeks, 4 days ago

Reviewed

0|i0620v:

ZooKeeper

ZOOKEEPER-16

Need to do path validation

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:17

13/Feb/09 16:18

30/Jan/09 20:27

3.0.0, 3.0.1

3.1.0

c client, java client, server

0

2

ZOOKEEPER-77

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1963141&group_id=209147&atid=1008544

48106

No Perforce job exists for this issue.

5

33260

16/Jun/08 3:22 PM

11 years, 8 weeks, 3 days ago

Reviewed

0|i06213:

ZooKeeper

ZOOKEEPER-15

handle failure better in build.xml:test

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:15

25/Oct/08 21:10

23/Jul/08 00:31

3.0.0

build

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1959060&group_id=209147&atid=1008544

48107

No Perforce job exists for this issue.

0

33261

11 years, 22 weeks, 4 days ago

0|i0621b:

ZooKeeper

ZOOKEEPER-14

Connect timeouts not calculated properly

Bug

Open

Major

Unresolved

Benjamin Reed

Patrick D. Hunt

10/Jun/08 17:14

03/Jan/12 09:38

c client, java client

0

1

ZOOKEEPER-843

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1948097&group_id=209147&atid=1008544

There should be a better connection between connect timeout, ping timeout, and session timeout. On the Java client it is sessionTimeout/hostsList.length. On the C client it's just the readTimeout. There is also an arbitrary delay when we find that all the hosts are down.

We should come up with a good consistent story for the connect timeout. Since we leave sessionTimeout/3 ms for recovery from a down server, it seems most reasonable to have something like sessionTimeout/(3*hostsList.length).

214024

No Perforce job exists for this issue.

0

33262

11 years, 42 weeks, 2 days ago

0|i0621j:

ZooKeeper

ZOOKEEPER-13

NPE during normal operations

Bug

Resolved

Major

Duplicate

Unassigned

Patrick D. Hunt

10/Jun/08 17:12

27/Aug/08 14:28

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1946934&group_id=209147&atid=1008544

214023

No Perforce job exists for this issue.

0

33263

11 years, 31 weeks, 1 day ago

0|i0621r:

ZooKeeper

ZOOKEEPER-12

getChildren doesn't return a list of paths

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 17:11

01/May/13 22:29

c client, documentation, java client

0

2

ZOOKEEPER-849

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1941120&group_id=209147&atid=1008544

api

62181

No Perforce job exists for this issue.

0

33264

29/Nov/10 3:52 AM

8 years, 29 weeks, 2 days ago

api

0|i0621z:

ZooKeeper

ZOOKEEPER-11

ArrayList is used instead of List

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 17:10

25/Oct/08 21:10

23/Jul/08 00:30

3.0.0

java client

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1941109&group_id=209147&atid=1008544

48108

No Perforce job exists for this issue.

0

33265

11 years, 22 weeks, 4 days ago

0|i06227:

ZooKeeper

ZOOKEEPER-10

Bad error message

Bug

Resolved

Minor

Invalid

Patrick D. Hunt

10/Jun/08 17:09

14/Mar/14 20:20

3.5.0

server

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1941108&group_id=209147&atid=1008544

70746

No Perforce job exists for this issue.

0

33266

14/Mar/14 8:20 PM

6 years, 1 week, 6 days ago

0|i0622f:

ZooKeeper

ZOOKEEPER-9

Set socket linger longer for commands

Bug

Open

Major

Unresolved

Unassigned

Patrick D. Hunt

10/Jun/08 17:07

06/Sep/11 12:14

server

0

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1888835&group_id=209147&atid=1008544

62180

No Perforce job exists for this issue.

0

33267

06/Sep/11 7:41 AM

8 years, 29 weeks, 2 days ago

0|i0622n:

ZooKeeper

ZOOKEEPER-8

Stat enchaned to include num of children and size

Bug

Closed

Minor

Fixed

Patrick D. Hunt

10/Jun/08 17:05

25/Oct/08 21:10

07/Oct/08 14:52

3.0.0

c client, java client

0

1

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1886743&group_id=209147&atid=1008544

48109

No Perforce job exists for this issue.

3

33268

02/Jul/08 2:37 PM

11 years, 22 weeks, 4 days ago

Incompatible change, Reviewed

0|i0622v:

ZooKeeper

ZOOKEEPER-7

Use enums rather than ints for types and state

Bug

Closed

Minor

Fixed

Jakob Homan

Patrick D. Hunt

10/Jun/08 17:01

25/Oct/08 21:10

26/Sep/08 17:27

3.0.0

java client

0

1

ZOOKEEPER-132, ZOOKEEPER-139

ZOOKEEPER-18

Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detail&aid=1831408&group_id=209147&atid=1008544

Would be nice to fix in 3.0 as it's a non-bw compat change, also user feedback show's confusion on state/event/missingjavadoc.

48110

No Perforce job exists for this issue.

0

33269

26/Sep/08 5:27 PM

11 years, 22 weeks, 4 days ago

Incompatible change

0|i06233:

ZooKeeper

ZOOKEEPER-6

List of problem identifiers in zookeeper.h

Bug

Closed

Major

Fixed

Patrick D. Hunt

10/Jun/08 16:42

25/Oct/08 21:10

30/Sep/08 07:15

3.0.0

c client

0

1

Submitted on behalf of Jacob. Is is possible for us to address this in the next release? Will cause b/w compatibility issues for c client users but sounds like a good idea to fix now.
--------
I've attached a file with the problem identifiers. All of these
identifiers are unprotected by a ZOO_ (or something similar) prefix.

I've also looked at zookeeper.jute.h and the names of some these structs
are really unfortunate -- theyre sure to collide with some headers e.g.
Stat. There's also some exceptions to the consistent naming scheme --
op_result_t, String_vector, Id_vector are the ones I noticed.

-------- file -------
These enum constants are unprotected:

typedef enum {LOG_LEVEL_ERROR=1,LOG_LEVEL_WARN=2,LOG_LEVEL_INFO=3,LOG_LEVEL_DEBUG=4} ZooLogLevel;

extern ZOOAPI const int PERM_READ;
extern ZOOAPI const int PERM_WRITE;
extern ZOOAPI const int PERM_CREATE;
extern ZOOAPI const int PERM_DELETE;
extern ZOOAPI const int PERM_ADMIN;
extern ZOOAPI const int PERM_ALL;

extern ZOOAPI struct Id ANYONE_ID_UNSAFE;
extern ZOOAPI struct Id AUTH_IDS;

extern ZOOAPI struct ACL_vector OPEN_ACL_UNSAFE;
extern ZOOAPI struct ACL_vector READ_ACL_UNSAFE;
extern ZOOAPI struct ACL_vector CREATOR_ALL_ACL;

extern ZOOAPI const int EPHEMERAL;
extern ZOOAPI const int SEQUENCE;

extern ZOOAPI const int EXPIRED_SESSION_STATE;
extern ZOOAPI const int AUTH_FAILED_STATE;
extern ZOOAPI const int CONNECTING_STATE;
extern ZOOAPI const int ASSOCIATING_STATE;
extern ZOOAPI const int CONNECTED_STATE;

extern ZOOAPI const int CREATED_EVENT;
extern ZOOAPI const int DELETED_EVENT;
extern ZOOAPI const int CHANGED_EVENT;
extern ZOOAPI const int CHILD_EVENT;
extern ZOOAPI const int SESSION_EVENT;
extern ZOOAPI const int NOTWATCHING_EVENT;

48111

No Perforce job exists for this issue.

1

33270

30/Sep/08 7:15 AM

11 years, 22 weeks, 4 days ago

Incompatible change

0|i0623b:

ZooKeeper

ZOOKEEPER-5

Upgrade Feature in Zookeeper server.

New Feature

Closed

Major

Fixed

Mahadev Konar

09/Jun/08 19:43

30/Mar/11 15:28

16/Oct/08 20:24

3.0.0

server

0

We need an upgrade feature in zookeeper where we can upgrade the old databases to a new one.

48112

No Perforce job exists for this issue.

10

33630

16/Oct/08 3:31 PM

9 years, 1 day ago

Reviewed

0|i064bb:

ZooKeeper

ZOOKEEPER-4

Unnecessary condition check in FastLeaderElection

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Benjamin Reed

09/Jun/08 12:42

14/Oct/08 16:31

09/Sep/08 17:09

3.0.0

leaderElection

0

2

FastLeaderElection.java line 224: The part of the condition after && is not needed: This is the else branch of an if statement, where the condition is exactly the first part. Hence, the part after && *must* be true.

48113

No Perforce job exists for this issue.

0

33271

09/Sep/08 5:09 PM

11 years, 29 weeks, 2 days ago

0|i0623j:

ZooKeeper

ZOOKEEPER-3

syncLimit has slightly different comments in the class header, and > inline with the variable.

Bug

Closed

Trivial

Fixed

Mahadev Konar

Benjamin Reed

09/Jun/08 12:39

09/Apr/10 09:43

18/Nov/09 12:48

3.3.0

quorum

0

3

syncLimit as documented in QuorumPeer is documented twice with two different aspects of in each instance. It should be better documented and unified. (Probably remove the second instance.)

48114

No Perforce job exists for this issue.

1

33272

16/Nov/09 4:08 PM

9 years, 50 weeks, 6 days ago

Reviewed

0|i0623r:

ZooKeeper

ZOOKEEPER-2

Synchronization issues in QuorumPeer and FastLeader election

Bug

Closed

Major

Fixed

Flavio Paiva Junqueira

Benjamin Reed

09/Jun/08 12:34

09/Sep/08 17:11

25/Aug/08 17:13

3.0.0

leaderElection

0

2

There are a couple of cases of member variables that need to be marked volatile or surrounded in a synchronization block. A couple of examples are:

* QuorumPeer state should be synchronous
* currentVote in QuorumPeer is marked volatile, but when it's members are often accessed individually as if they were in an atomic unit. Such code should be changed to get a reference to the currentVote and they access members through that reference.
* It looks like logicalClock in FastLeaderElection should be volatile. It should either be fixed or commented to explain why it doesn't need to be.

48115

No Perforce job exists for this issue.

3

33273

12/Jun/08 11:14 AM

11 years, 31 weeks ago

Reviewed

0|i0623z:

ZooKeeper

ZOOKEEPER-1

Initial ZooKeeper code contribution from Yahoo!

Task

Closed

Major

Fixed

Patrick D. Hunt

06/Jun/08 18:02

18/Mar/20 21:52

25/Oct/08 21:10

12/Jun/08 13:54

3.0.0

0

This is a SVN dump of the ZooKeeper repository (version 175) from SourceForge.

http://sourceforge.net/projects/zookeeper/

48116

No Perforce job exists for this issue.

1

33631

11 years, 22 weeks, 4 days ago

0|i064bj: